gowalk-cicd 1.0.125 → 1.0.127
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +15 -0
- package/README.md +20 -0
- package/action/.daemux-version +1 -1
- package/android-action/.daemux-version +1 -1
- package/android-action/play-upload/owner.cjs +31 -4
- package/backend-action/.daemux-version +1 -1
- package/backend-action/action.yml +36 -0
- package/backend-action/identity_diagnostics.py +202 -0
- package/backend-action/test_identity_diagnostics.py +101 -0
- package/backend-action/test_identity_failures.py +56 -0
- package/package.json +1 -1
- package/templates/deploy-backend.yml +14 -1
package/CLAUDE.md
CHANGED
|
@@ -419,3 +419,18 @@ node /path/to/gowalk-cicd/bin/cli.mjs
|
|
|
419
419
|
complete exact provider readback/recovery through scoped tooling, then reconcile its receipt.
|
|
420
420
|
Actions without `MOBILE_RELEASE_DEPLOYMENT_ID` preserve legacy consumer behavior. All new
|
|
421
421
|
receipt helper reads are fixed-origin GitHub requests; Apple/Google still use their assigned exits.
|
|
422
|
+
|
|
423
|
+
## Scoped backend diagnosis
|
|
424
|
+
|
|
425
|
+
The existing backend action accepts `operation: identity-diagnostics` with a UTC `diagnostic-since`
|
|
426
|
+
from the last24hours and exact `diagnostic-service` (default`api`). It performs no deploy, rsync,
|
|
427
|
+
Compose mutation or environment read. `identity_diagnostics.py` uses the existing CI SSH binding,
|
|
428
|
+
selects exactly one running container by Compose project/service, rechecks both labels and reads
|
|
429
|
+
at most1000recent log lines within a1MiB/30second bound. Raw output stays on the host; only the
|
|
430
|
+
fixed `firebase_identity_refused` stage/reason allowlist leaves it. No match means no observed
|
|
431
|
+
label, not successful authentication. Dispatch this action through the app's existing backend
|
|
432
|
+
workflow after a protected package release/adoption; never run hand-SSH or expose raw logs.
|
|
433
|
+
|
|
434
|
+
The diagnostic reader accepts the product-independent standard WARNING logger envelope, discards its
|
|
435
|
+
logger name and rejects trailing fields. Structured remote failure codes survive exit1; SSH transport,
|
|
436
|
+
read bounds and scope refusals have a closed vocabulary. Unknown exceptions remain unclassified.
|
package/README.md
CHANGED
|
@@ -1212,3 +1212,23 @@ with HTTP Range; if the origin ignores Range, a bounded full restart is permitte
|
|
|
1212
1212
|
must match the selected official manifest's SHA-256 before extraction. Authentication and certificate
|
|
1213
1213
|
refusals remain terminal. Exhausted transfers emit `flutter_download_failed` with fixed status codes and
|
|
1214
1214
|
retained-byte counts, without URLs or private curl stderr. Cache identities remain those of the pinned action.
|
|
1215
|
+
|
|
1216
|
+
### Read fixed backend identity refusal labels
|
|
1217
|
+
|
|
1218
|
+
The installed Backend Deploy workflow offers `operation=identity-diagnostics` on manual dispatch.
|
|
1219
|
+
Set `diagnostic-since` to a UTC `YYYY-MM-DDTHH:MM:SSZ` within the previous 24 hours.
|
|
1220
|
+
This uses the existing backend action and SSH binding to inspect the exact app's `api` Compose service.
|
|
1221
|
+
Custom workflows can set `diagnostic-service` explicitly. Retain the generated operation binding and
|
|
1222
|
+
skip app-specific runtime preparation when this mode is selected; preserve all normal deployment inputs.
|
|
1223
|
+
|
|
1224
|
+
The action performs no deployment, file sync, database query or runtime-configuration read. It returns
|
|
1225
|
+
`gowalk-cicd/backend-identity-diagnostics.v1` with `ok`, a closed `failure` code (empty on success), and at most100 `{stage,reason}` records from the
|
|
1226
|
+
fixed Firebase refusal vocabulary. Raw logs stay on the host. Missing/ambiguous containers, scope mismatch,
|
|
1227
|
+
transport failure or read bounds fail closed. An empty successful record list means no matching observation;
|
|
1228
|
+
it does not prove authentication succeeded. Match the bounded observation window to the owned client attempt.
|
|
1229
|
+
|
|
1230
|
+
Reader failures retain only: `invalid_scope`, `invalid_window`, `invalid_destination`,
|
|
1231
|
+
`container_ambiguous_or_absent`, `container_scope_mismatch`, `command_timeout`, `output_limit`,
|
|
1232
|
+
`command_failed`, `transport_failure`, `remote_invalid`, or `unclassified`. Structured remote refusals
|
|
1233
|
+
survive a remote exit1. SSH transport failures remain distinct. Unknown exceptions never contribute text.
|
|
1234
|
+
The optional standard `WARNING:<logger>:` prefix is discarded; logger names and trailing fields are never returned.
|
package/action/.daemux-version
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.127
|
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.127
|
|
@@ -12,18 +12,39 @@ function evidence(status) {
|
|
|
12
12
|
return 'release_write_status=error';
|
|
13
13
|
}
|
|
14
14
|
|
|
15
|
+
// Does the record name THIS run? That one question settles both halves of the
|
|
16
|
+
// protocol -- a claim is won when the answer is yes, released when it is no --
|
|
17
|
+
// and the answer does not depend on which write produced it.
|
|
18
|
+
//
|
|
19
|
+
// Both halves used to demand their OWN revision back from the read that
|
|
20
|
+
// follows their write. GitHub's contents API is read-after-write eventual on a
|
|
21
|
+
// branch ref, so that read can serve the previous blob for a moment, and a
|
|
22
|
+
// write that had already landed then looked like a failure. Foundy run
|
|
23
|
+
// 34345638488 (2026-09-09) released ownership twice -- both commits are on
|
|
24
|
+
// app-robot-play-coordination, both wrote `owner: null` -- and still failed
|
|
25
|
+
// `play_owner_release_unverified`, taking the app's first Play release with it.
|
|
26
|
+
const held = (row, selected) => record.same(row.value.owner, selected.owner);
|
|
27
|
+
|
|
28
|
+
// Seal owes two things, not one: the record must not name this run, AND no
|
|
29
|
+
// claim write of ours may still be able to land. A record another holder has
|
|
30
|
+
// taken settles both -- their write moved the blob, so a late CAS of ours can
|
|
31
|
+
// only conflict. An UNOWNED record settles only the first, so seal still writes
|
|
32
|
+
// once to move the revision itself, and then accepts any readback that no
|
|
33
|
+
// longer names this run. (That fence used to be a side effect of releasing a
|
|
34
|
+
// record we never held; a lost claim response relies on it.)
|
|
35
|
+
const fenced = (row, selected) => row.value.owner !== null && !held(row, selected);
|
|
36
|
+
|
|
15
37
|
async function seal(client, selected) {
|
|
16
38
|
let detail = 'release_write_status=none';
|
|
17
39
|
for (let attempt = 0; attempt < 3; attempt++) {
|
|
18
40
|
const row = record.validate(await client.read(selected.path), selected);
|
|
19
|
-
if (
|
|
41
|
+
if (fenced(row, selected)) return;
|
|
20
42
|
const idle = record.replacement(row, null);
|
|
21
43
|
// Reconcile a possibly applied write; keep only its status as evidence.
|
|
22
44
|
try { detail = evidence(await client.write(selected.path, row, idle)); }
|
|
23
45
|
catch { detail = evidence(undefined); }
|
|
24
46
|
const after = record.validate(await client.read(selected.path), selected);
|
|
25
|
-
if (after
|
|
26
|
-
if (after.value.owner !== null && !record.same(after.value.owner, selected.owner)) return;
|
|
47
|
+
if (!held(after, selected)) return;
|
|
27
48
|
}
|
|
28
49
|
const failure = new Error('play_owner_release_unverified');
|
|
29
50
|
failure.detail = detail;
|
|
@@ -40,6 +61,12 @@ async function acquire(client, selected, attempted = () => {}, waiting = {}) {
|
|
|
40
61
|
for (let attempt = 0; attempt < 3;) {
|
|
41
62
|
if (waiting.signal?.aborted) throw new Error('play_owner_cancelled');
|
|
42
63
|
const row = record.validate(await client.read(selected.path), selected);
|
|
64
|
+
// A record already naming THIS run is this run's own claim: the owner id is
|
|
65
|
+
// a UUID minted once per process, so no other writer can have put it there.
|
|
66
|
+
// Without this, a claim whose write landed but whose readback lagged came
|
|
67
|
+
// back on the next pass as a foreign live holder and the run waited out its
|
|
68
|
+
// own deadline against itself.
|
|
69
|
+
if (held(row, selected)) return;
|
|
43
70
|
if (row.value.owner !== null) {
|
|
44
71
|
const owner = row.value.owner;
|
|
45
72
|
if (owner.kind !== 'ci' || !record.terminal(await client.run(owner), owner)) {
|
|
@@ -55,7 +82,7 @@ async function acquire(client, selected, attempted = () => {}, waiting = {}) {
|
|
|
55
82
|
attempted();
|
|
56
83
|
try { await client.write(selected.path, row, desired); } catch { /* Only exact readback may adopt it. */ }
|
|
57
84
|
const after = record.validate(await client.read(selected.path), selected);
|
|
58
|
-
if (
|
|
85
|
+
if (held(after, selected)) return;
|
|
59
86
|
}
|
|
60
87
|
throw new Error('play_owner_busy');
|
|
61
88
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.127
|
|
@@ -6,6 +6,18 @@ description: >
|
|
|
6
6
|
The deploy host is publicly reachable, so this runs on a GitHub-hosted runner.
|
|
7
7
|
|
|
8
8
|
inputs:
|
|
9
|
+
operation:
|
|
10
|
+
description: deploy (default) or identity-diagnostics (read-only fixed Firebase refusal labels).
|
|
11
|
+
required: false
|
|
12
|
+
default: deploy
|
|
13
|
+
diagnostic-service:
|
|
14
|
+
description: Exact Compose service to inspect; used only for identity-diagnostics.
|
|
15
|
+
required: false
|
|
16
|
+
default: api
|
|
17
|
+
diagnostic-since:
|
|
18
|
+
description: UTC YYYY-MM-DDTHH:MM:SSZ within the previous 24 hours; required for diagnostics.
|
|
19
|
+
required: false
|
|
20
|
+
default: ""
|
|
9
21
|
host:
|
|
10
22
|
description: SSH host of the deploy server.
|
|
11
23
|
required: false
|
|
@@ -56,7 +68,15 @@ inputs:
|
|
|
56
68
|
runs:
|
|
57
69
|
using: composite
|
|
58
70
|
steps:
|
|
71
|
+
- name: Validate operation
|
|
72
|
+
shell: bash
|
|
73
|
+
env:
|
|
74
|
+
OPERATION: ${{ inputs.operation }}
|
|
75
|
+
run: |
|
|
76
|
+
case "$OPERATION" in deploy|identity-diagnostics) ;; *) exit 1;; esac
|
|
77
|
+
|
|
59
78
|
- name: Validate inputs
|
|
79
|
+
if: inputs.operation == 'deploy'
|
|
60
80
|
id: layout
|
|
61
81
|
shell: bash
|
|
62
82
|
env:
|
|
@@ -102,6 +122,7 @@ runs:
|
|
|
102
122
|
chmod 644 ~/.ssh/known_hosts
|
|
103
123
|
|
|
104
124
|
- name: Sync backend to the host
|
|
125
|
+
if: inputs.operation == 'deploy'
|
|
105
126
|
shell: bash
|
|
106
127
|
env:
|
|
107
128
|
BACKEND_DIR: ${{ inputs.backend-dir }}
|
|
@@ -139,6 +160,7 @@ runs:
|
|
|
139
160
|
"${{ inputs.ssh-user }}@${{ inputs.host }}:$DEST/"
|
|
140
161
|
|
|
141
162
|
- name: Deploy on the host
|
|
163
|
+
if: inputs.operation == 'deploy'
|
|
142
164
|
shell: bash
|
|
143
165
|
env:
|
|
144
166
|
RUNTIME_ENV: ${{ inputs.runtime-env }}
|
|
@@ -159,6 +181,20 @@ runs:
|
|
|
159
181
|
'${{ inputs.app-name }}' '${{ inputs.api-domain }}' \
|
|
160
182
|
'${{ inputs.health-path }}' '${{ inputs.cert-email }}'"
|
|
161
183
|
|
|
184
|
+
- name: Read fixed backend identity diagnostics
|
|
185
|
+
if: inputs.operation == 'identity-diagnostics'
|
|
186
|
+
shell: bash
|
|
187
|
+
env:
|
|
188
|
+
DIAGNOSTIC_APP: ${{ inputs.app-name }}
|
|
189
|
+
DIAGNOSTIC_SERVICE: ${{ inputs.diagnostic-service }}
|
|
190
|
+
DIAGNOSTIC_SINCE: ${{ inputs.diagnostic-since }}
|
|
191
|
+
DIAGNOSTIC_HOST: ${{ inputs.host }}
|
|
192
|
+
DIAGNOSTIC_USER: ${{ inputs.ssh-user }}
|
|
193
|
+
run: |
|
|
194
|
+
python3 "${{ github.action_path }}/identity_diagnostics.py" \
|
|
195
|
+
--app "$DIAGNOSTIC_APP" --service "$DIAGNOSTIC_SERVICE" --since "$DIAGNOSTIC_SINCE" \
|
|
196
|
+
--host "$DIAGNOSTIC_HOST" --user "$DIAGNOSTIC_USER"
|
|
197
|
+
|
|
162
198
|
- name: Cleanup key
|
|
163
199
|
if: always()
|
|
164
200
|
shell: bash
|
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Read one app's fixed Firebase refusal labels; raw host output never leaves this process."""
|
|
3
|
+
import argparse
|
|
4
|
+
import datetime
|
|
5
|
+
import json
|
|
6
|
+
import os
|
|
7
|
+
from pathlib import Path
|
|
8
|
+
import re
|
|
9
|
+
import selectors
|
|
10
|
+
import shlex
|
|
11
|
+
import signal
|
|
12
|
+
import subprocess
|
|
13
|
+
import time
|
|
14
|
+
|
|
15
|
+
STAGES = frozenset({'auth_signature', 'app_check_signature', 'app_scope', 'account_lookup', 'unclassified'})
|
|
16
|
+
REASONS = frozenset({
|
|
17
|
+
'unclassified', 'transport_timeout', 'transport_failure', 'token_invalid',
|
|
18
|
+
'app_scope_invalid', 'token_size_invalid', 'token_header_invalid', 'token_time_invalid',
|
|
19
|
+
'authentication_time_invalid', 'app_audience_invalid', 'subject_invalid', 'invalid_key_id',
|
|
20
|
+
'key_fetch_busy', 'key_refresh_limited', 'unknown_key', 'key_provider_unavailable',
|
|
21
|
+
'key_response_limit', 'key_set_invalid', 'key_identity_invalid', 'key_algorithm_invalid',
|
|
22
|
+
'key_strength_invalid', 'account_verification_refused', 'account_response_limit', 'account_missing',
|
|
23
|
+
'account_disabled_or_mismatched', 'account_revocation_boundary_missing', 'account_revoked',
|
|
24
|
+
})
|
|
25
|
+
SCHEMA = 'gowalk-cicd/backend-identity-diagnostics.v1'
|
|
26
|
+
LABEL = re.compile(r'(?:WARNING:[A-Za-z_][A-Za-z0-9_.-]{0,127}:)?'
|
|
27
|
+
r'firebase_identity_refused stage=([a-z_]+) reason=([a-z_]+)')
|
|
28
|
+
LIMIT = 1024 * 1024
|
|
29
|
+
FAILURES = frozenset({
|
|
30
|
+
'unclassified', 'invalid_scope', 'invalid_window', 'invalid_destination',
|
|
31
|
+
'container_ambiguous_or_absent', 'container_scope_mismatch', 'command_timeout',
|
|
32
|
+
'output_limit', 'command_failed', 'transport_failure', 'remote_invalid',
|
|
33
|
+
})
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
class ReaderFailure(Exception):
|
|
37
|
+
def __init__(self, code):
|
|
38
|
+
self.code = code if isinstance(code, str) and code in FAILURES else 'unclassified'
|
|
39
|
+
super().__init__(self.code)
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
def transfer(selector, process, payload, offset, data):
|
|
43
|
+
for key, _ in selector.select(0.1):
|
|
44
|
+
if key.fileobj is process.stdin:
|
|
45
|
+
offset += os.write(process.stdin.fileno(), payload[offset:offset + 4096])
|
|
46
|
+
if offset == len(payload):
|
|
47
|
+
selector.unregister(process.stdin)
|
|
48
|
+
process.stdin.close()
|
|
49
|
+
else:
|
|
50
|
+
chunk = os.read(process.stdout.fileno(), 65536)
|
|
51
|
+
if not chunk:
|
|
52
|
+
selector.unregister(process.stdout)
|
|
53
|
+
data.extend(chunk)
|
|
54
|
+
if len(data) > LIMIT:
|
|
55
|
+
raise ReaderFailure('output_limit')
|
|
56
|
+
return offset
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
def bounded(argv, *, payload=None, seconds=30, with_status=False):
|
|
60
|
+
"""Bound output, runtime and owned process cleanup without printing stderr."""
|
|
61
|
+
process = subprocess.Popen(argv, stdin=subprocess.PIPE if payload else subprocess.DEVNULL,
|
|
62
|
+
stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, start_new_session=True)
|
|
63
|
+
deadline, data = time.monotonic() + seconds, bytearray()
|
|
64
|
+
try:
|
|
65
|
+
with selectors.DefaultSelector() as selector:
|
|
66
|
+
selector.register(process.stdout, selectors.EVENT_READ)
|
|
67
|
+
if payload:
|
|
68
|
+
os.set_blocking(process.stdin.fileno(), False)
|
|
69
|
+
selector.register(process.stdin, selectors.EVENT_WRITE)
|
|
70
|
+
offset = 0
|
|
71
|
+
while selector.get_map():
|
|
72
|
+
if time.monotonic() >= deadline:
|
|
73
|
+
raise ReaderFailure('command_timeout')
|
|
74
|
+
offset = transfer(selector, process, payload, offset, data)
|
|
75
|
+
try:
|
|
76
|
+
status = process.wait(timeout=max(0.01, deadline - time.monotonic()))
|
|
77
|
+
except subprocess.TimeoutExpired:
|
|
78
|
+
raise ReaderFailure('command_timeout') from None
|
|
79
|
+
if with_status:
|
|
80
|
+
return status, bytes(data)
|
|
81
|
+
if status:
|
|
82
|
+
raise ReaderFailure('command_failed')
|
|
83
|
+
return bytes(data)
|
|
84
|
+
finally:
|
|
85
|
+
# Descendants can retain a pipe after the direct child exits. Kill only this owned group.
|
|
86
|
+
try:
|
|
87
|
+
os.killpg(process.pid, signal.SIGKILL)
|
|
88
|
+
except ProcessLookupError:
|
|
89
|
+
pass
|
|
90
|
+
process.wait(timeout=5)
|
|
91
|
+
process.stdout.close()
|
|
92
|
+
if process.stdin and not process.stdin.closed:
|
|
93
|
+
process.stdin.close()
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
def validate(app, service, since):
|
|
97
|
+
if not re.fullmatch(r'[a-z0-9][a-z0-9-]{0,79}', app):
|
|
98
|
+
raise ReaderFailure('invalid_scope')
|
|
99
|
+
if not re.fullmatch(r'[a-z0-9][a-z0-9_-]{0,63}', service):
|
|
100
|
+
raise ReaderFailure('invalid_scope')
|
|
101
|
+
if not re.fullmatch(r'\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ', since):
|
|
102
|
+
raise ReaderFailure('invalid_window')
|
|
103
|
+
try:
|
|
104
|
+
start = datetime.datetime.strptime(since, '%Y-%m-%dT%H:%M:%SZ').replace(tzinfo=datetime.timezone.utc)
|
|
105
|
+
except ValueError:
|
|
106
|
+
raise ReaderFailure('invalid_window') from None
|
|
107
|
+
age = (datetime.datetime.now(datetime.timezone.utc) - start).total_seconds()
|
|
108
|
+
if age < 0 or age > 86400:
|
|
109
|
+
raise ReaderFailure('invalid_window')
|
|
110
|
+
|
|
111
|
+
|
|
112
|
+
def records(raw):
|
|
113
|
+
result = []
|
|
114
|
+
for line in raw.decode('utf-8', errors='replace').splitlines():
|
|
115
|
+
match = LABEL.fullmatch(line)
|
|
116
|
+
if match and match[1] in STAGES and match[2] in REASONS:
|
|
117
|
+
result.append({'stage': match[1], 'reason': match[2]})
|
|
118
|
+
return result[-100:]
|
|
119
|
+
|
|
120
|
+
|
|
121
|
+
def remote(app, service, since):
|
|
122
|
+
validate(app, service, since)
|
|
123
|
+
ids = bounded(['docker', 'ps', '--no-trunc', '--filter', f'label=com.docker.compose.project={app}',
|
|
124
|
+
'--filter', f'label=com.docker.compose.service={service}', '--format', '{{.ID}}'], seconds=10).splitlines()
|
|
125
|
+
if len(ids) != 1 or not re.fullmatch(rb'[a-f0-9]{64}', ids[0]):
|
|
126
|
+
raise ReaderFailure('container_ambiguous_or_absent')
|
|
127
|
+
container = ids[0].decode('ascii')
|
|
128
|
+
# Recheck exact labels; Docker filter matching alone does not prove the requested scope.
|
|
129
|
+
inspected = json.loads(bounded(['docker', 'inspect', '--format', '{{json .Config.Labels}}', container],
|
|
130
|
+
seconds=10))
|
|
131
|
+
if (inspected.get('com.docker.compose.project') != app
|
|
132
|
+
or inspected.get('com.docker.compose.service') != service):
|
|
133
|
+
raise ReaderFailure('container_scope_mismatch')
|
|
134
|
+
# docker logs writes the container's stderr to its own stderr. Merge only inside the bounded reader.
|
|
135
|
+
command = ['docker', 'logs', '--since', since, '--tail', '1000', container]
|
|
136
|
+
raw = bounded(['sh', '-c', 'exec "$@" 2>&1', 'diagnostics', *command], seconds=10)
|
|
137
|
+
return {'schema': SCHEMA, 'ok': True, 'failure': '', 'records': records(raw)}
|
|
138
|
+
|
|
139
|
+
|
|
140
|
+
def local(args):
|
|
141
|
+
validate(args.app, args.service, args.since)
|
|
142
|
+
if (not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9.-]{0,252}', args.host or '')
|
|
143
|
+
or not re.fullmatch(r'[a-z_][a-z0-9_-]{0,31}', args.user)):
|
|
144
|
+
raise ReaderFailure('invalid_destination')
|
|
145
|
+
command = shlex.join(['python3', '-', '--remote', '--app', args.app,
|
|
146
|
+
'--service', args.service, '--since', args.since])
|
|
147
|
+
try:
|
|
148
|
+
status, raw = bounded(['ssh', '-i', str(Path.home() / '.ssh/backend_deploy'), '-o', 'IdentitiesOnly=yes',
|
|
149
|
+
'-o', 'BatchMode=yes', '-o', 'ConnectTimeout=10', '-o', 'ServerAliveInterval=5',
|
|
150
|
+
'-o', 'ServerAliveCountMax=2', f'{args.user}@{args.host}', command],
|
|
151
|
+
payload=Path(__file__).read_bytes(), seconds=45, with_status=True)
|
|
152
|
+
except OSError:
|
|
153
|
+
raise ReaderFailure('transport_failure') from None
|
|
154
|
+
return decode_remote(status, raw)
|
|
155
|
+
|
|
156
|
+
|
|
157
|
+
def decode_remote(status, raw):
|
|
158
|
+
if status not in (0, 1):
|
|
159
|
+
raise ReaderFailure('transport_failure')
|
|
160
|
+
try:
|
|
161
|
+
result = json.loads(raw)
|
|
162
|
+
except (ValueError, TypeError):
|
|
163
|
+
raise ReaderFailure('remote_invalid') from None
|
|
164
|
+
if (not isinstance(result, dict) or set(result) != {'schema', 'ok', 'failure', 'records'}
|
|
165
|
+
or result['schema'] != SCHEMA or not isinstance(result['ok'], bool)):
|
|
166
|
+
raise ReaderFailure('remote_invalid')
|
|
167
|
+
if status == 1 and result['ok'] is False and result['records'] == []:
|
|
168
|
+
raise ReaderFailure(result['failure'])
|
|
169
|
+
if status or result['ok'] is not True or result['failure'] != '':
|
|
170
|
+
raise ReaderFailure('remote_invalid')
|
|
171
|
+
if not isinstance(result['records'], list) or len(result['records']) > 100:
|
|
172
|
+
raise ReaderFailure('remote_invalid')
|
|
173
|
+
for row in result['records']:
|
|
174
|
+
if (not isinstance(row, dict) or set(row) != {'stage', 'reason'}
|
|
175
|
+
or not isinstance(row['stage'], str) or row['stage'] not in STAGES
|
|
176
|
+
or not isinstance(row['reason'], str) or row['reason'] not in REASONS):
|
|
177
|
+
raise ReaderFailure('remote_invalid')
|
|
178
|
+
return result
|
|
179
|
+
|
|
180
|
+
|
|
181
|
+
def main():
|
|
182
|
+
parser = argparse.ArgumentParser(description=__doc__)
|
|
183
|
+
parser.add_argument('--remote', action='store_true')
|
|
184
|
+
parser.add_argument('--app', required=True)
|
|
185
|
+
parser.add_argument('--service', default='api')
|
|
186
|
+
parser.add_argument('--since', required=True)
|
|
187
|
+
parser.add_argument('--host')
|
|
188
|
+
parser.add_argument('--user', default='root')
|
|
189
|
+
args = parser.parse_args()
|
|
190
|
+
try:
|
|
191
|
+
result = remote(args.app, args.service, args.since) if args.remote else local(args)
|
|
192
|
+
except Exception as error:
|
|
193
|
+
# Only our closed vocabulary survives; provider and parser exceptions remain unclassified.
|
|
194
|
+
code = error.code if isinstance(error, ReaderFailure) else 'unclassified'
|
|
195
|
+
print(json.dumps({'schema': SCHEMA, 'ok': False, 'failure': code, 'records': []}))
|
|
196
|
+
return 1
|
|
197
|
+
print(json.dumps(result, separators=(',', ':')))
|
|
198
|
+
return 0
|
|
199
|
+
|
|
200
|
+
|
|
201
|
+
if __name__ == '__main__':
|
|
202
|
+
raise SystemExit(main())
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
"""Protocol and process-boundary checks for the deployed read-only diagnostic reader."""
|
|
2
|
+
import datetime
|
|
3
|
+
import importlib.util
|
|
4
|
+
import json
|
|
5
|
+
from pathlib import Path
|
|
6
|
+
import sys
|
|
7
|
+
import unittest
|
|
8
|
+
from unittest.mock import patch
|
|
9
|
+
|
|
10
|
+
SPEC = importlib.util.spec_from_file_location('reader', Path(__file__).with_name('identity_diagnostics.py'))
|
|
11
|
+
READER = importlib.util.module_from_spec(SPEC)
|
|
12
|
+
SPEC.loader.exec_module(READER)
|
|
13
|
+
SINCE = (datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(minutes=5)).strftime('%Y-%m-%dT%H:%M:%SZ')
|
|
14
|
+
CID = b'a' * 64
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
class IdentityDiagnostics(unittest.TestCase):
|
|
18
|
+
def test_exact_labels_only(self):
|
|
19
|
+
raw = b'\n'.join([
|
|
20
|
+
b'firebase_identity_refused stage=auth_signature reason=token_invalid',
|
|
21
|
+
b'WARNING:cast_tv.firebase:firebase_identity_refused stage=account_lookup reason=account_revoked',
|
|
22
|
+
b'WARNING:second_product.identity:firebase_identity_refused stage=app_check_signature reason=unknown_key',
|
|
23
|
+
b'Bearer PRIVATE_SENTINEL',
|
|
24
|
+
b'firebase_identity_refused stage=PRIVATE_SENTINEL reason=token_invalid',
|
|
25
|
+
b'firebase_identity_refused stage=auth_signature reason=PRIVATE_SENTINEL',
|
|
26
|
+
b'firebase_identity_refused stage=auth_signature reason=token_invalid user=PRIVATE_SENTINEL',
|
|
27
|
+
b'prefix firebase_identity_refused stage=auth_signature reason=token_invalid',
|
|
28
|
+
])
|
|
29
|
+
self.assertEqual(READER.records(raw), [
|
|
30
|
+
{'stage': 'auth_signature', 'reason': 'token_invalid'},
|
|
31
|
+
{'stage': 'account_lookup', 'reason': 'account_revoked'},
|
|
32
|
+
{'stage': 'app_check_signature', 'reason': 'unknown_key'},
|
|
33
|
+
])
|
|
34
|
+
|
|
35
|
+
def test_recent_rows_bounded(self):
|
|
36
|
+
self.assertEqual(len(READER.records(
|
|
37
|
+
b'firebase_identity_refused stage=app_check_signature reason=unknown_key\n' * 1001)), 100)
|
|
38
|
+
|
|
39
|
+
def test_invalid_scopes_and_windows(self):
|
|
40
|
+
for app, service, since in [('../other', 'api', SINCE), ('app', 'api;id', SINCE),
|
|
41
|
+
('app', 'api', '2020-01-01T00:00:00Z'),
|
|
42
|
+
('app', 'api', '2099-01-01T00:00:00Z'), ('app', 'api', '10m')]:
|
|
43
|
+
with self.subTest(app=app, service=service, since=since), self.assertRaises(READER.ReaderFailure):
|
|
44
|
+
READER.validate(app, service, since)
|
|
45
|
+
|
|
46
|
+
def test_remote_exact_container_recheck_and_fixed_projection(self):
|
|
47
|
+
labels = json.dumps({'com.docker.compose.project': 'fixture', 'com.docker.compose.service': 'api'}).encode()
|
|
48
|
+
with patch.object(READER, 'bounded', side_effect=[CID + b'\n', labels,
|
|
49
|
+
b'PRIVATE_SENTINEL\nfirebase_identity_refused stage=app_scope reason=app_scope_invalid\n']) as run:
|
|
50
|
+
result = READER.remote('fixture', 'api', SINCE)
|
|
51
|
+
self.assertEqual(result['records'], [{'stage': 'app_scope', 'reason': 'app_scope_invalid'}])
|
|
52
|
+
commands = [call.args[0] for call in run.call_args_list]
|
|
53
|
+
self.assertEqual(commands[0], ['docker', 'ps', '--no-trunc', '--filter',
|
|
54
|
+
'label=com.docker.compose.project=fixture', '--filter', 'label=com.docker.compose.service=api',
|
|
55
|
+
'--format', '{{.ID}}'])
|
|
56
|
+
self.assertEqual(commands[1][-1], CID.decode())
|
|
57
|
+
self.assertEqual(commands[2][4:], ['docker', 'logs', '--since', SINCE, '--tail', '1000', CID.decode()])
|
|
58
|
+
self.assertNotIn('PRIVATE_SENTINEL', json.dumps(result))
|
|
59
|
+
|
|
60
|
+
def test_absent_ambiguous_and_foreign_containers_refuse(self):
|
|
61
|
+
for ids in [b'', CID + b'\n' + CID, b'PRIVATE_SENTINEL']:
|
|
62
|
+
with patch.object(READER, 'bounded', return_value=ids) as run, self.assertRaises(READER.ReaderFailure):
|
|
63
|
+
READER.remote('fixture', 'api', SINCE)
|
|
64
|
+
self.assertEqual(run.call_count, 1)
|
|
65
|
+
for project, service in [('other', 'api'), ('fixture', 'db')]:
|
|
66
|
+
labels = json.dumps({'com.docker.compose.project': project, 'com.docker.compose.service': service})
|
|
67
|
+
with patch.object(READER, 'bounded', side_effect=[CID, labels]) as run, \
|
|
68
|
+
self.assertRaises(READER.ReaderFailure):
|
|
69
|
+
READER.remote('fixture', 'api', SINCE)
|
|
70
|
+
self.assertEqual(run.call_count, 2)
|
|
71
|
+
|
|
72
|
+
def test_local_rejects_raw_or_extended_remote_output(self):
|
|
73
|
+
args = type('Args', (), dict(app='fixture', service='api', since=SINCE, host='fixture.invalid', user='root'))()
|
|
74
|
+
for raw in [b'PRIVATE_SENTINEL', json.dumps({'schema': READER.SCHEMA, 'ok': True,
|
|
75
|
+
'records': [{'stage': 'auth_signature', 'reason': 'PRIVATE_SENTINEL'}]}).encode(),
|
|
76
|
+
json.dumps({'schema': READER.SCHEMA, 'ok': True, 'records': [], 'raw': 'PRIVATE_SENTINEL'}).encode()]:
|
|
77
|
+
with patch.object(READER, 'bounded', return_value=(0, raw)), self.assertRaises(READER.ReaderFailure):
|
|
78
|
+
READER.local(args)
|
|
79
|
+
|
|
80
|
+
def test_owned_subprocess_success_and_stderr_suppression(self):
|
|
81
|
+
self.assertEqual(READER.bounded([sys.executable, '-c',
|
|
82
|
+
'import sys; print("ok"); print("PRIVATE_SENTINEL", file=sys.stderr)']), b'ok\n')
|
|
83
|
+
|
|
84
|
+
def test_failed_subprocess_does_not_export_output(self):
|
|
85
|
+
with self.assertRaisesRegex(READER.ReaderFailure, '^command_failed$'):
|
|
86
|
+
READER.bounded([sys.executable, '-c', 'print("PRIVATE_SENTINEL"); raise SystemExit(7)'])
|
|
87
|
+
|
|
88
|
+
def test_output_limit_and_owned_pipe_cleanup(self):
|
|
89
|
+
with self.assertRaisesRegex(READER.ReaderFailure, '^output_limit$'):
|
|
90
|
+
READER.bounded([sys.executable, '-c', 'print("x" * 1048577)'])
|
|
91
|
+
with self.assertRaisesRegex(READER.ReaderFailure, '^command_timeout$'):
|
|
92
|
+
READER.bounded([sys.executable, '-c',
|
|
93
|
+
'import os,time; child=os.fork(); time.sleep(10) if child == 0 else None'], seconds=0.2)
|
|
94
|
+
|
|
95
|
+
def test_stdin_upload_is_bounded_and_complete(self):
|
|
96
|
+
self.assertEqual(READER.bounded([sys.executable, '-c',
|
|
97
|
+
'import sys; print(len(sys.stdin.buffer.read()))'], payload=b'x' * 200000), b'200000\n')
|
|
98
|
+
|
|
99
|
+
|
|
100
|
+
if __name__ == '__main__':
|
|
101
|
+
unittest.main()
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
"""Closed failure propagation through remote output, transport and the public CLI."""
|
|
2
|
+
from contextlib import redirect_stdout
|
|
3
|
+
import io
|
|
4
|
+
import json
|
|
5
|
+
import sys
|
|
6
|
+
import unittest
|
|
7
|
+
from unittest.mock import patch
|
|
8
|
+
from test_identity_diagnostics import READER, SINCE
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
class IdentityFailures(unittest.TestCase):
|
|
12
|
+
def test_every_fixed_remote_refusal_survives_the_ssh_exit(self):
|
|
13
|
+
args = type('Args', (), dict(app='fixture', service='api', since=SINCE, host='fixture.invalid', user='root'))()
|
|
14
|
+
for code in READER.FAILURES:
|
|
15
|
+
raw = json.dumps({'schema': READER.SCHEMA, 'ok': False, 'failure': code, 'records': []}).encode()
|
|
16
|
+
with self.subTest(code=code), patch.object(READER, 'bounded', return_value=(1, raw)) as run:
|
|
17
|
+
with self.assertRaises(READER.ReaderFailure) as failure:
|
|
18
|
+
READER.local(args)
|
|
19
|
+
self.assertEqual(failure.exception.code, code)
|
|
20
|
+
self.assertTrue(run.call_args.kwargs['with_status'])
|
|
21
|
+
|
|
22
|
+
def test_transport_and_unknown_refusals_never_export_raw_values(self):
|
|
23
|
+
for status, raw, expected in [(255, b'PRIVATE_SENTINEL', 'transport_failure'),
|
|
24
|
+
(1, b'PRIVATE_SENTINEL', 'remote_invalid'),
|
|
25
|
+
(1, json.dumps({'schema': READER.SCHEMA, 'ok': False,
|
|
26
|
+
'failure': 'PRIVATE_SENTINEL', 'records': []}).encode(), 'unclassified')]:
|
|
27
|
+
with self.assertRaises(READER.ReaderFailure) as failure:
|
|
28
|
+
READER.decode_remote(status, raw)
|
|
29
|
+
self.assertEqual(failure.exception.code, expected)
|
|
30
|
+
self.assertNotIn('PRIVATE_SENTINEL', str(failure.exception))
|
|
31
|
+
|
|
32
|
+
def test_nonzero_process_keeps_only_validated_structured_refusal(self):
|
|
33
|
+
payload = json.dumps({'schema': READER.SCHEMA, 'ok': False,
|
|
34
|
+
'failure': 'container_scope_mismatch', 'records': []})
|
|
35
|
+
status, raw = READER.bounded([sys.executable, '-c',
|
|
36
|
+
'import sys; print(sys.argv[1]); print("PRIVATE_SENTINEL", file=sys.stderr); sys.exit(1)', payload],
|
|
37
|
+
with_status=True)
|
|
38
|
+
with self.assertRaisesRegex(READER.ReaderFailure, '^container_scope_mismatch$'):
|
|
39
|
+
READER.decode_remote(status, raw)
|
|
40
|
+
self.assertNotIn(b'PRIVATE_SENTINEL', raw)
|
|
41
|
+
|
|
42
|
+
def test_cli_known_and_unknown_errors_produce_closed_failure_envelopes(self):
|
|
43
|
+
argv = ['reader', '--remote', '--app', 'fixture', '--service', 'api', '--since', SINCE]
|
|
44
|
+
for error, expected in [(READER.ReaderFailure('command_timeout'), 'command_timeout'),
|
|
45
|
+
(RuntimeError('PRIVATE_SENTINEL'), 'unclassified')]:
|
|
46
|
+
output = io.StringIO()
|
|
47
|
+
with patch.object(sys, 'argv', argv), patch.object(READER, 'remote', side_effect=error), \
|
|
48
|
+
redirect_stdout(output):
|
|
49
|
+
self.assertEqual(READER.main(), 1)
|
|
50
|
+
self.assertEqual(json.loads(output.getvalue()), {
|
|
51
|
+
'schema': READER.SCHEMA, 'ok': False, 'failure': expected, 'records': []})
|
|
52
|
+
self.assertNotIn('PRIVATE_SENTINEL', output.getvalue())
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
if __name__ == '__main__':
|
|
56
|
+
unittest.main()
|
package/package.json
CHANGED
|
@@ -14,12 +14,22 @@ on:
|
|
|
14
14
|
- ".github/actions/backend-app/**"
|
|
15
15
|
- "!.github/actions/backend-app/.daemux-version"
|
|
16
16
|
workflow_dispatch:
|
|
17
|
+
inputs:
|
|
18
|
+
operation:
|
|
19
|
+
type: choice
|
|
20
|
+
description: Deploy or read fixed identity refusal labels without changing the running app
|
|
21
|
+
default: deploy
|
|
22
|
+
options: [deploy, identity-diagnostics]
|
|
23
|
+
diagnostic-since:
|
|
24
|
+
type: string
|
|
25
|
+
description: For diagnostics, UTC YYYY-MM-DDTHH:MM:SSZ within the previous 24 hours
|
|
26
|
+
required: false
|
|
17
27
|
|
|
18
28
|
permissions:
|
|
19
29
|
contents: read
|
|
20
30
|
|
|
21
31
|
concurrency:
|
|
22
|
-
group: backend-deploy-${{ github.ref }}
|
|
32
|
+
group: backend-deploy-${{ github.ref }}-${{ inputs.operation || 'deploy' }}
|
|
23
33
|
cancel-in-progress: true
|
|
24
34
|
|
|
25
35
|
jobs:
|
|
@@ -31,6 +41,7 @@ jobs:
|
|
|
31
41
|
|
|
32
42
|
# Pick the source dir and either its nested compose file or a root one.
|
|
33
43
|
- id: dir
|
|
44
|
+
if: inputs.operation != 'identity-diagnostics'
|
|
34
45
|
run: |
|
|
35
46
|
if [ -f backend/docker-compose.yml ]; then
|
|
36
47
|
echo "path=backend" >> "$GITHUB_OUTPUT"
|
|
@@ -52,6 +63,8 @@ jobs:
|
|
|
52
63
|
- name: Deploy backend
|
|
53
64
|
uses: ./.github/actions/backend-app
|
|
54
65
|
with:
|
|
66
|
+
operation: ${{ inputs.operation || 'deploy' }}
|
|
67
|
+
diagnostic-since: ${{ inputs.diagnostic-since }}
|
|
55
68
|
ssh-key: ${{ secrets.BACKEND_DEPLOY_SSH_KEY }}
|
|
56
69
|
host: ${{ vars.BACKEND_DEPLOY_HOST || '138.197.36.107' }}
|
|
57
70
|
app-name: ${{ vars.BACKEND_APP_NAME || github.event.repository.name }}
|