gowalk-cicd 1.0.183 → 1.0.185
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +26 -6
- package/action/action.yml +19 -3
- package/action/scripts/apple_upload_handoff.cjs +5 -9
- package/action/scripts/apple_upload_handoff.py +4 -1
- package/action/scripts/apple_upload_handoff_history.cjs +29 -0
- package/action/scripts/apple_upload_proof.cjs +117 -0
- package/action/scripts/apple_upload_proof.py +159 -0
- package/action/scripts/apple_upload_recovery.cjs +14 -7
- package/action/scripts/apple_upload_recovery_evidence.cjs +38 -0
- package/action/scripts/apple_upload_verify.cjs +4 -20
- package/action/scripts/reconcile_apple_upload.cjs +7 -20
- package/action/scripts/upload_build.py +11 -11
- package/docs/CONFIGURATION.md +5 -1
- package/package.json +1 -1
- package/remedies.json +4 -4
- package/src/config/workflows.mjs +12 -6
- package/templates/apple-upload-recovery.yml +96 -0
- package/templates/history.json.gz +0 -0
package/README.md
CHANGED
|
@@ -25,6 +25,8 @@ Writes three core things into your repo:
|
|
|
25
25
|
|
|
26
26
|
It also writes:
|
|
27
27
|
|
|
28
|
+
- `.github/workflows/apple-upload-recovery.yml` — explicit verification-only recovery for a retained iOS upload;
|
|
29
|
+
it checks the original producer and Apple build before completing that deployment, without rebuilding or uploading
|
|
28
30
|
- `.github/workflows/mobile-candidate.yml` — shipping binaries for ready task PRs
|
|
29
31
|
(dispatch-only while candidate artifacts are off)
|
|
30
32
|
- `.github/workflows/warm-android-cache.yml` — only while Android candidates are enabled: after this
|
|
@@ -1225,7 +1227,17 @@ commits completed part transfers with `uploaded: true`, following Apple's
|
|
|
1225
1227
|
The IPA endpoint rejected an optional `sourceFileChecksums` SHA_256 object even though
|
|
1226
1228
|
the generic schema lists that algorithm; the uploader no longer sends that attribute.
|
|
1227
1229
|
A completed file is reused only when its content-derived filename and size match, and
|
|
1228
|
-
any returned file checksum is verified.
|
|
1230
|
+
any nonnull returned whole-file checksum is verified. An explicit
|
|
1231
|
+
`sourceFileChecksums.file: null` means that the optional whole-file checksum is absent;
|
|
1232
|
+
a composite checksum does not substitute for a whole-IPA digest.
|
|
1233
|
+
New receipts retain locally computed SHA-256 and
|
|
1234
|
+
MD5 so Apple's returned checksum can be compared with the exact transferred bytes.
|
|
1235
|
+
|
|
1236
|
+
An upload summary can remain `PROCESSING` after Apple exposes a valid eligible build.
|
|
1237
|
+
Completion then requires the unique matching iOS build, marketing version, app and bundle,
|
|
1238
|
+
alongside the completed upload asset and matching checksum. A missing, pending, failed or
|
|
1239
|
+
ambiguous build remains unconfirmed. The receipt retains `upload_resource_state` separately
|
|
1240
|
+
from the verified completion state.
|
|
1229
1241
|
|
|
1230
1242
|
A failed upload emits the typed `apple_build_upload_failed` error annotation with schema
|
|
1231
1243
|
`gowalk-cicd/apple-build-upload-failed.v2`: `category` (`ipa_invalid`,
|
|
@@ -1235,13 +1247,17 @@ A failed upload emits the typed `apple_build_upload_failed` error annotation wit
|
|
|
1235
1247
|
the receipt's own identifiers (`cfBundleVersion`, `upload_id`, `file_id`, `state`,
|
|
1236
1248
|
`provider_status`, `provider_code`, `provider_pointer`). Exception text is never
|
|
1237
1249
|
published. The job also retains the exact exported IPA as `apple-ipa-<run>-<attempt>`
|
|
1238
|
-
(7 days)
|
|
1250
|
+
(7 days), copied without byte changes to the safe archive member `app.ipa`, so punctuation
|
|
1251
|
+
in the exported filename cannot block retention. A continuation can complete that upload instead of rebuilding
|
|
1239
1252
|
different bytes under the same number. Build numbers come from `/builds`,
|
|
1240
1253
|
`/preReleaseVersions` **and** `/apps/{id}/buildUploads`, so an unfinished or refused
|
|
1241
1254
|
reservation is never reused; delete such a reservation only after reading it back.
|
|
1242
1255
|
|
|
1243
|
-
When an iOS
|
|
1244
|
-
`Retain Apple verification handoff` succeeded, version 1.0.
|
|
1256
|
+
When an iOS deployment failed or was cancelled **after** both `Build and transfer iOS binary`
|
|
1257
|
+
and `Retain Apple verification handoff` succeeded, version 1.0.185 provides the managed
|
|
1258
|
+
`apple-upload-recovery.yml` workflow. Dispatch it on the repository's default branch with the
|
|
1259
|
+
original `producer-run`, `producer-attempt`, `deployment-id` and `handoff-artifact-id` inputs.
|
|
1260
|
+
It calls the released helper with those four validated identifiers:
|
|
1245
1261
|
|
|
1246
1262
|
```bash
|
|
1247
1263
|
node .github/actions/swift-app/scripts/apple_upload_recovery.cjs \
|
|
@@ -1288,7 +1304,8 @@ digest option: it downloads the exact original artifact, verifies GitHub's archi
|
|
|
1288
1304
|
and accepts only its bounded `apple-verification.json` member. It authenticates the current
|
|
1289
1305
|
dispatch separately from the original run, attempt, source, repository, branch and deployment.
|
|
1290
1306
|
|
|
1291
|
-
Recovery requires a terminal
|
|
1307
|
+
Recovery requires a terminal original run: either cancelled with its cancelled iOS producer,
|
|
1308
|
+
or failed with a completed successful or failed iOS producer. Both paths require those two
|
|
1292
1309
|
successful steps in order. A later completed attempt may exist; an active latest attempt
|
|
1293
1310
|
refuses, including a retry observed immediately before writing the receipt. Fresh Apple
|
|
1294
1311
|
readback must prove the exact completed upload and file, matching bytes, eligible `VALID`
|
|
@@ -1298,10 +1315,13 @@ and the old handoff's metadata and release notes are never replayed. Missing, ex
|
|
|
1298
1315
|
mismatched evidence remains refused; do not relabel a receipt or impersonate the original
|
|
1299
1316
|
`GITHUB_*` environment. Normal delivery still refuses cancelled producers.
|
|
1300
1317
|
|
|
1318
|
+
The managed workflow retains `apple-recovered-<recovery-run>-<recovery-attempt>` with the
|
|
1319
|
+
unchanged `apple-verification.json`, its `.verified.json` proof and derived `.upload-receipt.json`.
|
|
1320
|
+
The proof identifies the original producer and the current recovery caller separately.
|
|
1301
1321
|
Retain the recovery proof, then rerun the **whole original deployment workflow**. Its
|
|
1302
1322
|
upload-once admission reuses the confirmed iOS receipt and skips the iOS binary. Follow that
|
|
1303
1323
|
new attempt and any unrelated remaining jobs to completion; recovery does not change the
|
|
1304
|
-
original cancelled attempt's conclusion. Read current store state for remaining submission
|
|
1324
|
+
original failed or cancelled attempt's conclusion. Read current store state for remaining submission
|
|
1305
1325
|
work without changing an already accepted submission. This exceptional command adds no
|
|
1306
1326
|
normal delivery job, polling or latency.
|
|
1307
1327
|
|
package/action/action.yml
CHANGED
|
@@ -825,17 +825,33 @@ runs:
|
|
|
825
825
|
# A failed REST upload keeps the exact exported IPA next to its receipt, so a
|
|
826
826
|
# continuation can resume or complete that same upload identity instead of
|
|
827
827
|
# rebuilding different bytes under the number the receipt names.
|
|
828
|
-
- name:
|
|
828
|
+
- name: Stage the exported IPA for upload recovery
|
|
829
|
+
id: ipa-recovery
|
|
829
830
|
if: ${{ always() && inputs.archive == 'true' && inputs.upload == 'true'
|
|
830
831
|
&& (steps.upload.outcome != 'success' || inputs.defer-provider-verification == 'true') }}
|
|
832
|
+
shell: bash
|
|
833
|
+
run: |
|
|
834
|
+
shopt -s nullglob
|
|
835
|
+
ipas=("$RUNNER_TEMP"/export/*.ipa)
|
|
836
|
+
if [ "${#ipas[@]}" -ne 1 ]; then
|
|
837
|
+
echo '::error::apple_ipa_recovery_export_count_invalid'
|
|
838
|
+
exit 1
|
|
839
|
+
fi
|
|
840
|
+
retained="$RUNNER_TEMP/apple-ipa-recovery"
|
|
841
|
+
mkdir -p "$retained"
|
|
842
|
+
cp "${ipas[0]}" "$retained/app.ipa"
|
|
843
|
+
cmp -s "${ipas[0]}" "$retained/app.ipa"
|
|
844
|
+
|
|
845
|
+
- name: Retain the exported IPA for upload recovery
|
|
846
|
+
if: ${{ always() && steps.ipa-recovery.outcome == 'success' }}
|
|
831
847
|
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4
|
|
832
848
|
env:
|
|
833
849
|
NODE_OPTIONS: --require "${{ env.SWIFT_APP_ACTION }}/scripts/artifact_env.cjs"
|
|
834
850
|
with:
|
|
835
851
|
name: apple-ipa-${{ github.run_id }}-${{ github.run_attempt }}
|
|
836
|
-
path: ${{ runner.temp }}/
|
|
852
|
+
path: ${{ runner.temp }}/apple-ipa-recovery/app.ipa
|
|
837
853
|
compression-level: 0 # An IPA is already a ZIP; avoid duplicate compression on the Mac.
|
|
838
|
-
if-no-files-found:
|
|
854
|
+
if-no-files-found: error
|
|
839
855
|
retention-days: 7
|
|
840
856
|
|
|
841
857
|
# The whatsNew backstop. An empty "What's New" makes a version
|
|
@@ -4,6 +4,7 @@ const fs = require('node:fs');
|
|
|
4
4
|
const {createHash} = require('node:crypto');
|
|
5
5
|
const {join} = require('node:path');
|
|
6
6
|
const github = require('./mobile_release_api.cjs');
|
|
7
|
+
const {historicalProducer, uniqueArtifact} = require('./apple_upload_handoff_history.cjs');
|
|
7
8
|
const SCHEMA = 'gowalk-cicd/apple-upload-handoff.v1';
|
|
8
9
|
const positive = value => Number.isSafeInteger(value) && value > 0;
|
|
9
10
|
const id = value => typeof value === 'string' && /^[A-Za-z0-9-]{1,64}$/.test(value);
|
|
@@ -24,6 +25,7 @@ function validate(value) {
|
|
|
24
25
|
!/^[a-f0-9]{64}$/.test(r.sha256 || '') || r.fileName !== `${r.sha256}.ipa` || !positive(r.fileSize)) {
|
|
25
26
|
fail('handoff_receipt');
|
|
26
27
|
}
|
|
28
|
+
if (r.md5 !== undefined && (typeof r.md5 !== 'string' || !/^[a-f0-9]{32}$/.test(r.md5))) fail('handoff_receipt');
|
|
27
29
|
if (typeof m.enabled !== 'boolean' || typeof m.only_if_empty !== 'boolean' ||
|
|
28
30
|
typeof m.notes !== 'string' || Buffer.byteLength(m.notes) > 1024 * 1024 || m.notes.includes('\0') ||
|
|
29
31
|
!/^[A-Za-z0-9-]{2,35}$/.test(m.locale || '') || m.enabled && !id(m.version_id)) fail('handoff_metadata');
|
|
@@ -63,26 +65,20 @@ async function verifyProducer(value, artifactId, api = github, recoverCancelled
|
|
|
63
65
|
run.path !== '.github/workflows/deploy.yml' || run.event !== o.event ||
|
|
64
66
|
`refs/heads/${run.head_branch}` !== o.ref || run.repository?.full_name !== o.repository ||
|
|
65
67
|
run.head_repository?.full_name !== o.repository) fail('producer_run');
|
|
66
|
-
if (recoverCancelled)
|
|
67
|
-
const latest = await api.request('GET', `/actions/runs/${o.run_id}`);
|
|
68
|
-
if (run.status !== 'completed' || run.conclusion !== 'cancelled' || latest.id !== o.run_id ||
|
|
69
|
-
latest.status !== 'completed' || !positive(latest.run_attempt) || latest.run_attempt < o.run_attempt ||
|
|
70
|
-
latest.head_sha !== o.source_sha || latest.path !== run.path || latest.event !== o.event ||
|
|
71
|
-
latest.head_branch !== run.head_branch || latest.repository?.full_name !== o.repository ||
|
|
72
|
-
latest.head_repository?.full_name !== o.repository) fail('cancelled_producer_run');
|
|
73
|
-
}
|
|
68
|
+
if (recoverCancelled) await historicalProducer(o, run, api, fail);
|
|
74
69
|
const result = await api.request('GET', `/actions/runs/${o.run_id}/attempts/${o.run_attempt}/jobs?per_page=100`);
|
|
75
70
|
if (!Array.isArray(result.jobs) || result.jobs.length >= 100) fail('producer_jobs');
|
|
76
71
|
const jobs = result.jobs.filter(job => job.name === 'iOS TestFlight' || job.name?.endsWith('/ iOS TestFlight'));
|
|
77
72
|
// A later symbol-retention failure does not undo these exact completed steps.
|
|
78
73
|
// The producer's original failure remains red; only Apple delivery is reconciled here.
|
|
79
|
-
const conclusions = recoverCancelled ? ['cancelled'] : ['success', 'failure'];
|
|
74
|
+
const conclusions = recoverCancelled && run.conclusion === 'cancelled' ? ['cancelled'] : ['success', 'failure'];
|
|
80
75
|
if (jobs.length !== 1 || jobs[0].status !== 'completed' || !conclusions.includes(jobs[0].conclusion) ||
|
|
81
76
|
jobs[0].head_sha !== o.source_sha || !transferred(jobs[0])) fail('producer_job');
|
|
82
77
|
const artifact = await api.request('GET', `/actions/artifacts/${artifactId}`);
|
|
83
78
|
if (artifact.id !== artifactId || artifact.expired !== false || artifact.workflow_run?.id !== o.run_id ||
|
|
84
79
|
artifact.workflow_run?.head_sha !== o.source_sha ||
|
|
85
80
|
artifact.name !== `apple-verification-${o.run_id}-${o.run_attempt}`) fail('producer_artifact');
|
|
81
|
+
if (recoverCancelled) await uniqueArtifact(o, artifact, api, fail);
|
|
86
82
|
}
|
|
87
83
|
|
|
88
84
|
async function phase(value, api = github) {
|
|
@@ -54,7 +54,10 @@ def prepare():
|
|
|
54
54
|
def write(path, handoff, receipt):
|
|
55
55
|
if receipt.get("stage") != "transferred" or "\n" in str(path) or "\r" in str(path):
|
|
56
56
|
raise ValueError("apple_handoff_transfer_unconfirmed")
|
|
57
|
-
|
|
57
|
+
fields = {name: receipt[name] for name in FIELDS}
|
|
58
|
+
if "md5" in receipt:
|
|
59
|
+
fields["md5"] = receipt["md5"]
|
|
60
|
+
handoff = {**handoff, "receipt": fields}
|
|
58
61
|
data = (json.dumps(handoff, sort_keys=True, separators=(",", ":")) + "\n").encode()
|
|
59
62
|
path.write_bytes(data)
|
|
60
63
|
digest = hashlib.sha256(data).hexdigest()
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
/* Recheck one terminal historical producer and its unique retained handoff. */
|
|
2
|
+
'use strict';
|
|
3
|
+
const positive = value => Number.isSafeInteger(value) && value > 0;
|
|
4
|
+
|
|
5
|
+
async function historicalProducer(owner, run, api, fail) {
|
|
6
|
+
const latest = await api.request('GET', `/actions/runs/${owner.run_id}`);
|
|
7
|
+
const terminal = ['success', 'failure', 'cancelled', 'timed_out', 'action_required',
|
|
8
|
+
'neutral', 'skipped', 'stale', 'startup_failure'];
|
|
9
|
+
if (run.status !== 'completed' || !['cancelled', 'failure'].includes(run.conclusion) ||
|
|
10
|
+
latest.id !== owner.run_id || latest.status !== 'completed' || !terminal.includes(latest.conclusion) ||
|
|
11
|
+
!positive(latest.run_attempt) || latest.run_attempt < owner.run_attempt ||
|
|
12
|
+
latest.run_attempt === owner.run_attempt && latest.conclusion !== run.conclusion ||
|
|
13
|
+
latest.head_sha !== owner.source_sha || latest.path !== run.path || latest.event !== owner.event ||
|
|
14
|
+
latest.head_branch !== run.head_branch || latest.repository?.full_name !== owner.repository ||
|
|
15
|
+
latest.head_repository?.full_name !== owner.repository) fail('historical_producer_run');
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
async function uniqueArtifact(owner, artifact, api, fail) {
|
|
19
|
+
const result = await api.request('GET', `/actions/runs/${owner.run_id}/artifacts?per_page=100`);
|
|
20
|
+
if (!Array.isArray(result.artifacts) || result.artifacts.length >= 100 ||
|
|
21
|
+
!Number.isSafeInteger(result.total_count) || result.total_count !== result.artifacts.length) fail('producer_artifact');
|
|
22
|
+
const selected = result.artifacts.filter(row => row.name === artifact.name);
|
|
23
|
+
if (selected.length !== 1 || selected[0].id !== artifact.id || selected[0].expired !== false ||
|
|
24
|
+
selected[0].workflow_run?.id !== owner.run_id || selected[0].workflow_run?.head_sha !== owner.source_sha ||
|
|
25
|
+
!/^sha256:[a-f0-9]{64}$/.test(artifact.digest || '') || selected[0].digest !== artifact.digest ||
|
|
26
|
+
!positive(artifact.size_in_bytes) || selected[0].size_in_bytes !== artifact.size_in_bytes) fail('producer_artifact');
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
module.exports = {historicalProducer, uniqueArtifact};
|
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
/* Read-only proof of one transferred IPA, including Apple's separately processed build. */
|
|
2
|
+
'use strict';
|
|
3
|
+
const identifier = value => typeof value === 'string' && /^[A-Za-z0-9-]{1,64}$/.test(value);
|
|
4
|
+
|
|
5
|
+
function entity(response, fail, code) {
|
|
6
|
+
const row = response?.data;
|
|
7
|
+
if (!row || typeof row !== 'object' || Array.isArray(row) ||
|
|
8
|
+
row.attributes !== undefined && (!row.attributes || typeof row.attributes !== 'object') ||
|
|
9
|
+
row.relationships !== undefined && (!row.relationships || typeof row.relationships !== 'object')) fail(code);
|
|
10
|
+
return row;
|
|
11
|
+
}
|
|
12
|
+
|
|
13
|
+
function collection(response, fail, code) {
|
|
14
|
+
const rows = response?.data, total = response?.meta?.paging?.total;
|
|
15
|
+
if (!Array.isArray(rows) || rows.some(row => !row || typeof row !== 'object' || Array.isArray(row)) ||
|
|
16
|
+
rows.length >= 200 || !response.links || typeof response.links !== 'object' ||
|
|
17
|
+
Array.isArray(response.links) || response.links.next ||
|
|
18
|
+
total !== undefined && (!Number.isSafeInteger(total) || total !== rows.length)) fail(code);
|
|
19
|
+
return rows;
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
function checksum(attributes, receipt, fail) {
|
|
23
|
+
const checksums = attributes.sourceFileChecksums;
|
|
24
|
+
if (checksums !== undefined && (!checksums || typeof checksums !== 'object' || Array.isArray(checksums))) {
|
|
25
|
+
fail('file_checksum_unverified');
|
|
26
|
+
}
|
|
27
|
+
const file = checksums?.file;
|
|
28
|
+
if (file === undefined || file === null) return;
|
|
29
|
+
const digest = file?.algorithm === 'SHA_256' ? receipt.sha256 :
|
|
30
|
+
file?.algorithm === 'MD5' ? receipt.md5 : null;
|
|
31
|
+
const width = file?.algorithm === 'SHA_256' ? 64 : 32;
|
|
32
|
+
if (typeof digest !== 'string' || !new RegExp(`^[a-f0-9]{${width}}$`).test(digest)) {
|
|
33
|
+
fail('file_checksum_unverified');
|
|
34
|
+
}
|
|
35
|
+
const values = [digest, digest.toUpperCase(), Buffer.from(digest, 'hex').toString('base64')];
|
|
36
|
+
if (!values.includes(file.hash)) fail('file_checksum_unverified');
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
async function fileProof(receipt, get, fail) {
|
|
40
|
+
const response = await get(`/buildUploads/${receipt.upload_id}/buildUploadFiles`);
|
|
41
|
+
const assets = collection(response, fail, 'file_membership').filter(row => row.attributes?.assetType === 'ASSET');
|
|
42
|
+
if (assets.length !== 1 || assets[0].id !== receipt.file_id) fail('file_membership');
|
|
43
|
+
const file = entity(await get(`/buildUploadFiles/${receipt.file_id}`), fail, 'file_unverified');
|
|
44
|
+
const attrs = file.attributes || {};
|
|
45
|
+
if (file?.id !== receipt.file_id || attrs.assetType !== 'ASSET' ||
|
|
46
|
+
attrs.fileName !== receipt.fileName || attrs.fileSize !== receipt.fileSize) fail('file_unverified');
|
|
47
|
+
if (attrs.assetDeliveryState?.state === 'FAILED' || attrs.assetDeliveryState?.errors?.length) {
|
|
48
|
+
fail('processing_failed');
|
|
49
|
+
}
|
|
50
|
+
if (attrs.assetDeliveryState?.state !== 'COMPLETE') return false;
|
|
51
|
+
checksum(attrs, receipt, fail);
|
|
52
|
+
return true;
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
async function inventoryBuild(receipt, get, fail) {
|
|
56
|
+
const query = new URLSearchParams({'filter[app]': receipt.app_id, 'filter[version]': receipt.cfBundleVersion,
|
|
57
|
+
include: 'preReleaseVersion,app', limit: '200'});
|
|
58
|
+
const response = await get(`/builds?${query}`), rows = collection(response, fail, 'build_inventory_unverified');
|
|
59
|
+
const included = response.included;
|
|
60
|
+
if (!Array.isArray(included) || included.some(row => !row || typeof row !== 'object' || Array.isArray(row))) {
|
|
61
|
+
fail('build_inventory_unverified');
|
|
62
|
+
}
|
|
63
|
+
const ids = new Set(), matches = [];
|
|
64
|
+
for (const row of rows) {
|
|
65
|
+
if (!identifier(row.id) || ids.has(row.id) || row.attributes?.version !== receipt.cfBundleVersion) {
|
|
66
|
+
fail('build_inventory_unverified');
|
|
67
|
+
}
|
|
68
|
+
ids.add(row.id);
|
|
69
|
+
const trainId = row.relationships?.preReleaseVersion?.data?.id;
|
|
70
|
+
const appId = row.relationships?.app?.data?.id;
|
|
71
|
+
const trains = included.filter(item => item.type === 'preReleaseVersions' && item.id === trainId);
|
|
72
|
+
const apps = included.filter(item => item.type === 'apps' && item.id === appId);
|
|
73
|
+
if (!identifier(trainId) || trains.length !== 1 || apps.length !== 1 || appId !== receipt.app_id ||
|
|
74
|
+
apps[0].attributes?.bundleId !== receipt.bundle_id) fail('build_inventory_unverified');
|
|
75
|
+
if (trains[0].attributes?.version === receipt.cfBundleShortVersionString &&
|
|
76
|
+
trains[0].attributes?.platform === 'IOS') matches.push(row.id);
|
|
77
|
+
}
|
|
78
|
+
if (matches.length > 1) fail('build_ambiguous');
|
|
79
|
+
return matches[0] || null;
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
async function buildProof(buildId, receipt, get, fail) {
|
|
83
|
+
const build = entity(await get(`/builds/${buildId}`), fail, 'build_unverified');
|
|
84
|
+
const attrs = build.attributes || {};
|
|
85
|
+
if (build?.id !== buildId || attrs.version !== receipt.cfBundleVersion) fail('build_unverified');
|
|
86
|
+
if (['FAILED', 'INVALID'].includes(attrs.processingState)) fail('processing_failed');
|
|
87
|
+
if (attrs.processingState === 'PROCESSING') return null;
|
|
88
|
+
if (attrs.processingState !== 'VALID' || attrs.buildAudienceType !== 'APP_STORE_ELIGIBLE') {
|
|
89
|
+
fail('build_unverified');
|
|
90
|
+
}
|
|
91
|
+
const app = entity(await get(`/builds/${buildId}/app`), fail, 'app_unverified');
|
|
92
|
+
if (app?.id !== receipt.app_id || app.attributes?.bundleId !== receipt.bundle_id) fail('app_unverified');
|
|
93
|
+
const train = entity(await get(`/builds/${buildId}/preReleaseVersion`), fail, 'train_unverified');
|
|
94
|
+
if (!identifier(train?.id) || train.attributes?.platform !== 'IOS' ||
|
|
95
|
+
train.attributes?.version !== receipt.cfBundleShortVersionString) fail('train_unverified');
|
|
96
|
+
return {build_id: buildId, app_id: app.id, version: receipt.cfBundleShortVersionString,
|
|
97
|
+
build: receipt.cfBundleVersion, upload_id: receipt.upload_id};
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
async function observe(receipt, get, fail) {
|
|
101
|
+
const upload = entity(await get(`/buildUploads/${receipt.upload_id}?include=build`), fail, 'upload_unverified');
|
|
102
|
+
const attrs = upload?.attributes || {}, state = attrs.state?.state;
|
|
103
|
+
if (upload?.id !== receipt.upload_id || attrs.platform !== 'IOS' ||
|
|
104
|
+
attrs.cfBundleVersion !== receipt.cfBundleVersion ||
|
|
105
|
+
attrs.cfBundleShortVersionString !== receipt.cfBundleShortVersionString) fail('upload_unverified');
|
|
106
|
+
if (state === 'FAILED' || attrs.state?.errors?.length) fail('processing_failed');
|
|
107
|
+
if (!['COMPLETE', 'PROCESSING'].includes(state)) return null;
|
|
108
|
+
if (!await fileProof(receipt, get, fail)) return null;
|
|
109
|
+
const relationship = upload.relationships?.build?.data;
|
|
110
|
+
if (relationship && !identifier(relationship.id)) fail('build_unverified');
|
|
111
|
+
const buildId = relationship?.id || await inventoryBuild(receipt, get, fail);
|
|
112
|
+
if (!buildId) return null;
|
|
113
|
+
const proof = await buildProof(buildId, receipt, get, fail);
|
|
114
|
+
return proof && {...proof, upload_resource_state: state};
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
module.exports = {observe};
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
"""Read-only exact upload/file/build proof for the legacy synchronous Python entrypoint."""
|
|
2
|
+
import base64
|
|
3
|
+
import re
|
|
4
|
+
|
|
5
|
+
from upload_receipt import UploadError
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
def refuse(code):
|
|
9
|
+
raise UploadError(code, "Apple upload completion was not independently verified")
|
|
10
|
+
|
|
11
|
+
|
|
12
|
+
def entity(response, code):
|
|
13
|
+
row = response.get("data") if isinstance(response, dict) else None
|
|
14
|
+
if (not isinstance(row, dict) or not isinstance(row.get("attributes", {}), dict)
|
|
15
|
+
or not isinstance(row.get("relationships", {}), dict)):
|
|
16
|
+
refuse(code)
|
|
17
|
+
return row
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
def collection(response, code):
|
|
21
|
+
if not isinstance(response, dict):
|
|
22
|
+
refuse(code)
|
|
23
|
+
rows, links = response.get("data"), response.get("links")
|
|
24
|
+
total = (response.get("meta") or {}).get("paging", {}).get("total")
|
|
25
|
+
if (not isinstance(rows, list) or any(not isinstance(row, dict)
|
|
26
|
+
or not isinstance(row.get("attributes", {}), dict)
|
|
27
|
+
or not isinstance(row.get("relationships", {}), dict) for row in rows)
|
|
28
|
+
or len(rows) >= 200 or not isinstance(links, dict) or links.get("next")
|
|
29
|
+
or total is not None and (type(total) is not int or total != len(rows))):
|
|
30
|
+
refuse(code)
|
|
31
|
+
return rows
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def checksum(attributes, receipt):
|
|
35
|
+
checksums = attributes.get("sourceFileChecksums", {})
|
|
36
|
+
if not isinstance(checksums, dict):
|
|
37
|
+
refuse("file_checksum_unverified")
|
|
38
|
+
if "file" not in checksums:
|
|
39
|
+
return
|
|
40
|
+
file = checksums["file"]
|
|
41
|
+
if file is None:
|
|
42
|
+
return
|
|
43
|
+
if not isinstance(file, dict):
|
|
44
|
+
refuse("file_checksum_unverified")
|
|
45
|
+
algorithm = file.get("algorithm")
|
|
46
|
+
digest = receipt.get("sha256") if algorithm == "SHA_256" else receipt.get("md5") if algorithm == "MD5" else None
|
|
47
|
+
width = 64 if algorithm == "SHA_256" else 32
|
|
48
|
+
if not isinstance(digest, str) or not re.fullmatch(r"[a-f0-9]{" + str(width) + "}", digest):
|
|
49
|
+
refuse("file_checksum_unverified")
|
|
50
|
+
values = (digest, digest.upper(), base64.b64encode(bytes.fromhex(digest)).decode())
|
|
51
|
+
if file.get("hash") not in values:
|
|
52
|
+
refuse("file_checksum_unverified")
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
def file_proof(receipt, api):
|
|
56
|
+
response = api("GET", f"/buildUploads/{receipt['upload_id']}/buildUploadFiles")
|
|
57
|
+
assets = [row for row in collection(response, "file_membership")
|
|
58
|
+
if (row.get("attributes") or {}).get("assetType") == "ASSET"]
|
|
59
|
+
if len(assets) != 1 or assets[0].get("id") != receipt["file_id"]:
|
|
60
|
+
refuse("file_membership")
|
|
61
|
+
file = entity(api("GET", f"/buildUploadFiles/{receipt['file_id']}"), "file_unverified")
|
|
62
|
+
attrs = file.get("attributes") or {}
|
|
63
|
+
if (file.get("id") != receipt["file_id"] or attrs.get("assetType") != "ASSET"
|
|
64
|
+
or any(attrs.get(key) != receipt[key] for key in ("fileName", "fileSize"))):
|
|
65
|
+
refuse("file_unverified")
|
|
66
|
+
state = attrs.get("assetDeliveryState") or {}
|
|
67
|
+
if not isinstance(state, dict):
|
|
68
|
+
refuse("file_unverified")
|
|
69
|
+
if state.get("state") == "FAILED" or state.get("errors"):
|
|
70
|
+
refuse("processing_failed")
|
|
71
|
+
if state.get("state") != "COMPLETE":
|
|
72
|
+
return False
|
|
73
|
+
checksum(attrs, receipt)
|
|
74
|
+
return True
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
def inventory_build(receipt, api):
|
|
78
|
+
response = api("GET", "/builds", params={"filter[app]": receipt["app_id"],
|
|
79
|
+
"filter[version]": receipt["cfBundleVersion"], "include": "preReleaseVersion,app", "limit": 200})
|
|
80
|
+
rows = collection(response, "build_inventory_unverified")
|
|
81
|
+
included = response.get("included")
|
|
82
|
+
if (not isinstance(included, list) or any(not isinstance(item, dict)
|
|
83
|
+
or not isinstance(item.get("attributes", {}), dict) for item in included)):
|
|
84
|
+
refuse("build_inventory_unverified")
|
|
85
|
+
identifiers, matches = set(), []
|
|
86
|
+
for row in rows:
|
|
87
|
+
build_id = row.get("id")
|
|
88
|
+
if (not isinstance(build_id, str) or not re.fullmatch(r"[A-Za-z0-9-]{1,64}", build_id)
|
|
89
|
+
or build_id in identifiers
|
|
90
|
+
or (row.get("attributes") or {}).get("version") != receipt["cfBundleVersion"]):
|
|
91
|
+
refuse("build_inventory_unverified")
|
|
92
|
+
identifiers.add(build_id)
|
|
93
|
+
relationships = row.get("relationships") or {}
|
|
94
|
+
for name in ("preReleaseVersion", "app"):
|
|
95
|
+
link = relationships.get(name)
|
|
96
|
+
if (not isinstance(link, dict) or not isinstance(link.get("data"), dict)):
|
|
97
|
+
refuse("build_inventory_unverified")
|
|
98
|
+
train_id = ((relationships.get("preReleaseVersion") or {}).get("data") or {}).get("id")
|
|
99
|
+
app_id = ((relationships.get("app") or {}).get("data") or {}).get("id")
|
|
100
|
+
trains = [item for item in included if item.get("type") == "preReleaseVersions" and item.get("id") == train_id]
|
|
101
|
+
apps = [item for item in included if item.get("type") == "apps" and item.get("id") == app_id]
|
|
102
|
+
if (not isinstance(train_id, str) or not re.fullmatch(r"[A-Za-z0-9-]{1,64}", train_id)
|
|
103
|
+
or len(trains) != 1 or len(apps) != 1 or app_id != receipt["app_id"]
|
|
104
|
+
or (apps[0].get("attributes") or {}).get("bundleId") != receipt["bundle_id"]):
|
|
105
|
+
refuse("build_inventory_unverified")
|
|
106
|
+
attrs = trains[0].get("attributes") or {}
|
|
107
|
+
if attrs.get("version") == receipt["cfBundleShortVersionString"] and attrs.get("platform") == "IOS":
|
|
108
|
+
matches.append(build_id)
|
|
109
|
+
if len(matches) > 1:
|
|
110
|
+
refuse("build_ambiguous")
|
|
111
|
+
return matches[0] if matches else None
|
|
112
|
+
|
|
113
|
+
|
|
114
|
+
def build_proof(build_id, receipt, api):
|
|
115
|
+
build = entity(api("GET", f"/builds/{build_id}"), "build_unverified")
|
|
116
|
+
attrs = build.get("attributes") or {}
|
|
117
|
+
if build.get("id") != build_id or attrs.get("version") != receipt["cfBundleVersion"]:
|
|
118
|
+
refuse("build_unverified")
|
|
119
|
+
if attrs.get("processingState") in ("FAILED", "INVALID"):
|
|
120
|
+
refuse("processing_failed")
|
|
121
|
+
if attrs.get("processingState") == "PROCESSING":
|
|
122
|
+
return None
|
|
123
|
+
if attrs.get("processingState") != "VALID" or attrs.get("buildAudienceType") != "APP_STORE_ELIGIBLE":
|
|
124
|
+
refuse("build_unverified")
|
|
125
|
+
app = entity(api("GET", f"/builds/{build_id}/app"), "app_unverified")
|
|
126
|
+
if app.get("id") != receipt["app_id"] or (app.get("attributes") or {}).get("bundleId") != receipt["bundle_id"]:
|
|
127
|
+
refuse("app_unverified")
|
|
128
|
+
train = entity(api("GET", f"/builds/{build_id}/preReleaseVersion"), "train_unverified")
|
|
129
|
+
train_attrs = train.get("attributes") or {}
|
|
130
|
+
if (not isinstance(train.get("id"), str) or not re.fullmatch(r"[A-Za-z0-9-]{1,64}", train["id"])
|
|
131
|
+
or train_attrs.get("platform") != "IOS"
|
|
132
|
+
or train_attrs.get("version") != receipt["cfBundleShortVersionString"]):
|
|
133
|
+
refuse("train_unverified")
|
|
134
|
+
return {"build_id": build_id}
|
|
135
|
+
|
|
136
|
+
|
|
137
|
+
def observe(receipt, api):
|
|
138
|
+
response = api("GET", f"/buildUploads/{receipt['upload_id']}", params={"include": "build"})
|
|
139
|
+
upload = entity(response, "upload_unverified")
|
|
140
|
+
attrs = upload.get("attributes") or {}
|
|
141
|
+
state = attrs.get("state") or {}
|
|
142
|
+
if not isinstance(state, dict):
|
|
143
|
+
refuse("upload_unverified")
|
|
144
|
+
if (upload.get("id") != receipt["upload_id"] or attrs.get("platform") != "IOS"
|
|
145
|
+
or any(attrs.get(key) != receipt[key] for key in ("cfBundleVersion", "cfBundleShortVersionString"))):
|
|
146
|
+
refuse("upload_unverified")
|
|
147
|
+
if state.get("state") == "FAILED" or state.get("errors"):
|
|
148
|
+
refuse("processing_failed")
|
|
149
|
+
if state.get("state") not in ("COMPLETE", "PROCESSING") or not file_proof(receipt, api):
|
|
150
|
+
return None, state.get("state")
|
|
151
|
+
link = (upload.get("relationships") or {}).get("build")
|
|
152
|
+
if link is not None and not isinstance(link, dict):
|
|
153
|
+
refuse("build_unverified")
|
|
154
|
+
relationship = (link or {}).get("data")
|
|
155
|
+
if relationship and (not isinstance(relationship, dict) or not isinstance(relationship.get("id"), str)
|
|
156
|
+
or not re.fullmatch(r"[A-Za-z0-9-]{1,64}", relationship["id"])):
|
|
157
|
+
refuse("build_unverified")
|
|
158
|
+
build_id = relationship["id"] if relationship else inventory_build(receipt, api)
|
|
159
|
+
return (build_proof(build_id, receipt, api) if build_id else None), state.get("state")
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
/* Explicit Actions recovery of a
|
|
1
|
+
/* Explicit Actions recovery of a terminal producer's authenticated handoff; never upload or metadata. */
|
|
2
2
|
'use strict';
|
|
3
3
|
const fs = require('node:fs');
|
|
4
4
|
const {createHash} = require('node:crypto');
|
|
@@ -6,6 +6,7 @@ const {inflateRawSync} = require('node:zlib');
|
|
|
6
6
|
const github = require('./mobile_release_api.cjs');
|
|
7
7
|
const handoff = require('./apple_upload_handoff.cjs');
|
|
8
8
|
const verifier = require('./apple_upload_verify.cjs');
|
|
9
|
+
const {pinnedArtifact, evidence} = require('./apple_upload_recovery_evidence.cjs');
|
|
9
10
|
const LIMIT = 2 * 1024 * 1024;
|
|
10
11
|
const positive = value => Number.isSafeInteger(value) && value > 0;
|
|
11
12
|
const fail = code => handoff.fail(`recovery_${code}`);
|
|
@@ -83,7 +84,8 @@ async function current(env, api) {
|
|
|
83
84
|
run.repository?.full_name !== repository || run.head_repository?.full_name !== repository ||
|
|
84
85
|
!/^\.github\/workflows\/[\w.-]+\.ya?ml$/.test(run.path || '') ||
|
|
85
86
|
env.GITHUB_WORKFLOW_REF !== `${repository}/${run.path}@${env.GITHUB_REF}`) fail('caller_unverified');
|
|
86
|
-
return {run_id: runId, run_attempt: attempt, repository, ref: env.GITHUB_REF
|
|
87
|
+
return {run_id: runId, run_attempt: attempt, repository, ref: env.GITHUB_REF,
|
|
88
|
+
source_sha: env.GITHUB_SHA, workflow_path: run.path, event: run.event};
|
|
87
89
|
}
|
|
88
90
|
|
|
89
91
|
async function recover({runId, attempt, deploymentId, artifactId}, dependencies = {}) {
|
|
@@ -91,7 +93,7 @@ async function recover({runId, attempt, deploymentId, artifactId}, dependencies
|
|
|
91
93
|
const env = dependencies.env || process.env, api = dependencies.api || github;
|
|
92
94
|
const caller = await current(env, api);
|
|
93
95
|
if (runId === caller.run_id) fail('caller_invalid');
|
|
94
|
-
const artifact = await api.request('GET', `/actions/artifacts/${artifactId}`);
|
|
96
|
+
const artifact = structuredClone(await api.request('GET', `/actions/artifacts/${artifactId}`));
|
|
95
97
|
if (artifact.id !== artifactId || artifact.expired !== false || artifact.workflow_run?.id !== runId ||
|
|
96
98
|
artifact.name !== `apple-verification-${runId}-${attempt}`) fail('artifact_identity');
|
|
97
99
|
const bytes = await download(artifact, env, dependencies.fetch || fetch);
|
|
@@ -101,20 +103,25 @@ async function recover({runId, attempt, deploymentId, artifactId}, dependencies
|
|
|
101
103
|
if (owner.repository !== caller.repository || owner.ref !== caller.ref || owner.run_id !== runId ||
|
|
102
104
|
owner.run_attempt !== attempt || owner.deployment_id !== deploymentId ||
|
|
103
105
|
owner.source_sha !== artifact.workflow_run?.head_sha) fail('handoff_owner');
|
|
104
|
-
const result = await verifier.verify(value, artifactId, 900,
|
|
105
|
-
|
|
106
|
+
const result = await verifier.verify(value, artifactId, 900,
|
|
107
|
+
{...dependencies, api: pinnedArtifact(api, artifact), recoverCancelled: true});
|
|
108
|
+
const proof = {...result, schema: 'gowalk-cicd/apple-upload-recovered.v1', producer_run: runId,
|
|
106
109
|
repository: owner.repository, producer_source_sha: owner.source_sha,
|
|
107
110
|
producer_ref: owner.ref, producer_event: owner.event,
|
|
108
111
|
artifact_id: artifactId, artifact_digest: artifact.digest,
|
|
109
112
|
handoff_sha256: createHash('sha256').update(bytes).digest('hex'),
|
|
110
113
|
ipa_sha256: value.receipt.sha256, file_id: value.receipt.file_id,
|
|
111
|
-
recovery_run: caller.run_id, recovery_attempt: caller.run_attempt
|
|
114
|
+
recovery_run: caller.run_id, recovery_attempt: caller.run_attempt,
|
|
115
|
+
recovery_source_sha: caller.source_sha, recovery_ref: caller.ref,
|
|
116
|
+
recovery_event: caller.event, recovery_workflow_path: caller.workflow_path};
|
|
117
|
+
if (dependencies.writeEvidence === true) Object.assign(proof, evidence(bytes, value, proof, env));
|
|
118
|
+
return proof;
|
|
112
119
|
}
|
|
113
120
|
|
|
114
121
|
async function main() {
|
|
115
122
|
if (process.argv.length !== 6) fail('arguments');
|
|
116
123
|
const [runId, attempt, deploymentId, artifactId] = process.argv.slice(2).map(Number);
|
|
117
|
-
console.log(JSON.stringify(await recover({runId, attempt, deploymentId, artifactId})));
|
|
124
|
+
console.log(JSON.stringify(await recover({runId, attempt, deploymentId, artifactId}, {writeEvidence: true})));
|
|
118
125
|
}
|
|
119
126
|
|
|
120
127
|
if (require.main === module) main().catch(error => {
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
/* Preserve the exact authenticated handoff and record separately owned verification evidence. */
|
|
2
|
+
'use strict';
|
|
3
|
+
const fs = require('node:fs');
|
|
4
|
+
const {join} = require('node:path');
|
|
5
|
+
const handoff = require('./apple_upload_handoff.cjs');
|
|
6
|
+
const fail = code => handoff.fail(`recovery_${code}`);
|
|
7
|
+
|
|
8
|
+
function pinnedArtifact(api, original) {
|
|
9
|
+
return {...api, request: async (method, path, ...args) => {
|
|
10
|
+
const result = await api.request(method, path, ...args);
|
|
11
|
+
if (path === `/actions/artifacts/${original.id}` && (result.id !== original.id ||
|
|
12
|
+
result.digest !== original.digest || result.size_in_bytes !== original.size_in_bytes ||
|
|
13
|
+
result.expired !== false || result.name !== original.name ||
|
|
14
|
+
result.workflow_run?.id !== original.workflow_run.id ||
|
|
15
|
+
result.workflow_run?.head_sha !== original.workflow_run.head_sha)) fail('artifact_changed');
|
|
16
|
+
return result;
|
|
17
|
+
}};
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
function evidence(bytes, value, proof, env) {
|
|
21
|
+
if (!env.RUNNER_TEMP || /[\r\n]/.test(env.RUNNER_TEMP)) fail('evidence_path');
|
|
22
|
+
const directory = join(env.RUNNER_TEMP, 'apple-upload-recovery');
|
|
23
|
+
fs.mkdirSync(directory, {recursive: true, mode: 0o700});
|
|
24
|
+
const original = join(directory, 'apple-verification.json');
|
|
25
|
+
const receipt = `${original}.upload-receipt.json`, verified = `${original}.verified.json`;
|
|
26
|
+
fs.writeFileSync(original, bytes, {mode: 0o600});
|
|
27
|
+
if (!fs.readFileSync(original).equals(bytes)) fail('evidence_identity');
|
|
28
|
+
fs.writeFileSync(receipt, JSON.stringify({...value.receipt, stage: 'complete', state: 'COMPLETE',
|
|
29
|
+
build_id: proof.build_id, upload_resource_state: proof.upload_resource_state,
|
|
30
|
+
processing_seconds: proof.processing_seconds}, null, 2) + '\n', {mode: 0o600});
|
|
31
|
+
fs.writeFileSync(verified, JSON.stringify(proof, null, 2) + '\n', {mode: 0o600});
|
|
32
|
+
if (env.GITHUB_OUTPUT) fs.appendFileSync(env.GITHUB_OUTPUT,
|
|
33
|
+
`verified=true\nbuild-id=${proof.build_id}\nartifact-directory=${directory}\n` +
|
|
34
|
+
`handoff-path=${original}\nreceipt-path=${receipt}\nverified-proof-path=${verified}\n`);
|
|
35
|
+
return {artifact_directory: directory, handoff_path: original, receipt_path: receipt, verified_proof_path: verified};
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
module.exports = {pinnedArtifact, evidence};
|
|
@@ -5,6 +5,7 @@ const {parseArgs} = require('node:util');
|
|
|
5
5
|
const {setTimeout: delay} = require('node:timers/promises');
|
|
6
6
|
const github = require('./mobile_release_api.cjs');
|
|
7
7
|
const apple = require('./reconcile_apple_upload.cjs');
|
|
8
|
+
const provider = require('./apple_upload_proof.cjs');
|
|
8
9
|
const handoff = require('./apple_upload_handoff.cjs');
|
|
9
10
|
|
|
10
11
|
function credential(env = process.env) {
|
|
@@ -15,25 +16,7 @@ function credential(env = process.env) {
|
|
|
15
16
|
}
|
|
16
17
|
|
|
17
18
|
async function observe(receipt, get) {
|
|
18
|
-
|
|
19
|
-
const upload = response.data, attrs = upload?.attributes || {};
|
|
20
|
-
if (upload?.id !== receipt.upload_id || attrs.cfBundleVersion !== receipt.cfBundleVersion ||
|
|
21
|
-
attrs.cfBundleShortVersionString !== receipt.cfBundleShortVersionString) handoff.fail('upload_identity');
|
|
22
|
-
if (attrs.state?.state === 'FAILED') handoff.fail('processing_failed');
|
|
23
|
-
if (attrs.state?.state !== 'COMPLETE') return null;
|
|
24
|
-
const buildId = upload.relationships?.build?.data?.id;
|
|
25
|
-
if (!/^[A-Za-z0-9-]{1,64}$/.test(buildId || '')) handoff.fail('build_identity');
|
|
26
|
-
const buildPath = `/builds/${buildId}`, build = await get(buildPath);
|
|
27
|
-
if (['FAILED', 'INVALID'].includes(build.data?.attributes?.processingState)) handoff.fail('processing_failed');
|
|
28
|
-
if (build.data?.attributes?.processingState === 'PROCESSING') return null;
|
|
29
|
-
const files = await get(`/buildUploads/${receipt.upload_id}/buildUploadFiles`);
|
|
30
|
-
const assets = files.data?.filter(row => row.attributes?.assetType === 'ASSET');
|
|
31
|
-
if (!Array.isArray(assets) || files.links?.next || assets.length !== 1 || assets[0].id !== receipt.file_id) {
|
|
32
|
-
handoff.fail('file_membership');
|
|
33
|
-
}
|
|
34
|
-
// Preserve the existing independent complete-file, VALID/eligible build and actual app readbacks.
|
|
35
|
-
return apple.verifyApple(receipt, requested => requested === path ? response :
|
|
36
|
-
requested === buildPath ? build : get(requested));
|
|
19
|
+
return provider.observe(receipt, get, handoff.fail);
|
|
37
20
|
}
|
|
38
21
|
|
|
39
22
|
async function verify(value, artifactId, timeout, dependencies = {}) {
|
|
@@ -71,7 +54,8 @@ async function main() {
|
|
|
71
54
|
handoff.metadata(value);
|
|
72
55
|
const receiptPath = `${positionals[0]}.upload-receipt.json`;
|
|
73
56
|
fs.writeFileSync(receiptPath, JSON.stringify({...value.receipt, stage: 'complete', state: 'COMPLETE',
|
|
74
|
-
build_id: proof.build_id,
|
|
57
|
+
build_id: proof.build_id, upload_resource_state: proof.upload_resource_state,
|
|
58
|
+
processing_seconds: proof.processing_seconds}, null, 2) + '\n');
|
|
75
59
|
if (/[\r\n]/.test(receiptPath)) handoff.fail('receipt_path');
|
|
76
60
|
if (process.env.GITHUB_OUTPUT) fs.appendFileSync(process.env.GITHUB_OUTPUT,
|
|
77
61
|
`verified=true\nbuild-id=${proof.build_id}\nproducer-attempt=${proof.producer_attempt}\n` +
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
const fs = require('node:fs');
|
|
4
4
|
const crypto = require('node:crypto');
|
|
5
5
|
const github = require('./mobile_release_api.cjs');
|
|
6
|
+
const provider = require('./apple_upload_proof.cjs');
|
|
6
7
|
|
|
7
8
|
const RECEIPT = 'gowalk-cicd/apple-build-upload-receipt.v1';
|
|
8
9
|
const OWNER = 'gowalk-cicd/mobile-upload-owner.v1';
|
|
@@ -20,6 +21,9 @@ function readReceipt(path) {
|
|
|
20
21
|
!/^\d+(\.\d+){1,2}$/.test(data.cfBundleShortVersionString || '') ||
|
|
21
22
|
!/^[a-f0-9]{64}$/.test(data.sha256 || '') || data.fileName !== `${data.sha256}.ipa` ||
|
|
22
23
|
!Number.isSafeInteger(data.fileSize) || data.fileSize <= 0) refuse('receipt_invalid');
|
|
24
|
+
if (data.md5 !== undefined && (typeof data.md5 !== 'string' || !/^[a-f0-9]{32}$/.test(data.md5))) {
|
|
25
|
+
refuse('receipt_invalid');
|
|
26
|
+
}
|
|
23
27
|
return data;
|
|
24
28
|
}
|
|
25
29
|
|
|
@@ -69,26 +73,9 @@ async function appleGet(path, key = {pem: process.env.ASC_KEY_P8, id: process.en
|
|
|
69
73
|
}
|
|
70
74
|
|
|
71
75
|
async function verifyApple(receipt, get = appleGet) {
|
|
72
|
-
const
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
if (upload?.id !== receipt.upload_id || attributes.state?.state !== 'COMPLETE' ||
|
|
76
|
-
attributes.cfBundleVersion !== receipt.cfBundleVersion ||
|
|
77
|
-
attributes.cfBundleShortVersionString !== receipt.cfBundleShortVersionString ||
|
|
78
|
-
!/^[A-Za-z0-9-]{1,64}$/.test(buildId || '')) refuse('upload_unverified');
|
|
79
|
-
const file = (await get(`/buildUploadFiles/${receipt.file_id}`)).data;
|
|
80
|
-
if (file?.id !== receipt.file_id || file.attributes?.assetType !== 'ASSET' ||
|
|
81
|
-
file.attributes.fileName !== receipt.fileName ||
|
|
82
|
-
file.attributes.fileSize !== receipt.fileSize ||
|
|
83
|
-
file.attributes.assetDeliveryState?.state !== 'COMPLETE') refuse('file_unverified');
|
|
84
|
-
const build = (await get(`/builds/${buildId}`)).data;
|
|
85
|
-
if (build?.id !== buildId || build.attributes?.processingState !== 'VALID' ||
|
|
86
|
-
build.attributes?.buildAudienceType !== 'APP_STORE_ELIGIBLE' ||
|
|
87
|
-
build.attributes?.version !== receipt.cfBundleVersion) refuse('build_unverified');
|
|
88
|
-
const app = (await get(`/builds/${buildId}/app`)).data;
|
|
89
|
-
if (app?.id !== receipt.app_id || app.attributes?.bundleId !== receipt.bundle_id) refuse('app_unverified');
|
|
90
|
-
return {build_id: buildId, app_id: app.id, version: receipt.cfBundleShortVersionString,
|
|
91
|
-
build: receipt.cfBundleVersion, upload_id: receipt.upload_id};
|
|
76
|
+
const proof = await provider.observe(receipt, get, refuse);
|
|
77
|
+
if (!proof) refuse('processing_pending');
|
|
78
|
+
return proof;
|
|
92
79
|
}
|
|
93
80
|
|
|
94
81
|
async function reconcile(deploymentId, runId, attempt, path, api = github, get = appleGet) {
|
|
@@ -18,6 +18,7 @@ import requests
|
|
|
18
18
|
import upload_checksums
|
|
19
19
|
import upload_receipt
|
|
20
20
|
import apple_upload_handoff
|
|
21
|
+
import apple_upload_proof
|
|
21
22
|
from upload_receipt import Receipt, UploadError
|
|
22
23
|
|
|
23
24
|
# Provider refusals the uploader classifies itself instead of letting the shared
|
|
@@ -53,12 +54,16 @@ def identity(ipa: Path) -> dict:
|
|
|
53
54
|
info = plistlib.loads(archive.read(names[0]))
|
|
54
55
|
except (zipfile.BadZipFile, plistlib.InvalidFileException, KeyError) as exc:
|
|
55
56
|
raise UploadError("ipa_invalid", "IPA is not a readable app archive") from exc
|
|
57
|
+
sha256, md5 = hashlib.sha256(), hashlib.md5()
|
|
56
58
|
with ipa.open("rb") as stream:
|
|
57
|
-
|
|
59
|
+
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
|
60
|
+
sha256.update(chunk)
|
|
61
|
+
md5.update(chunk)
|
|
62
|
+
digest = sha256.hexdigest()
|
|
58
63
|
return {"bundle_id": str(info["CFBundleIdentifier"]),
|
|
59
64
|
"cfBundleVersion": str(info["CFBundleVersion"]),
|
|
60
65
|
"cfBundleShortVersionString": str(info["CFBundleShortVersionString"]),
|
|
61
|
-
"sha256": digest, "fileName": digest + ".ipa", "fileSize": ipa.stat().st_size}
|
|
66
|
+
"sha256": digest, "md5": md5.hexdigest(), "fileName": digest + ".ipa", "fileSize": ipa.stat().st_size}
|
|
62
67
|
|
|
63
68
|
|
|
64
69
|
def upload_record(app_id: str, info: dict) -> dict:
|
|
@@ -176,17 +181,12 @@ def deliver(ipa: Path, report: Path, timeout: int, handoff: Path | None = None)
|
|
|
176
181
|
processing_started = time.monotonic()
|
|
177
182
|
deadline = processing_started + timeout
|
|
178
183
|
while True:
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
if state == "COMPLETE":
|
|
183
|
-
return receipt.update("complete", state=state, build_id=build_id,
|
|
184
|
+
proof, state = apple_upload_proof.observe(receipt.data, api)
|
|
185
|
+
if proof:
|
|
186
|
+
return receipt.update("complete", state="COMPLETE", upload_resource_state=state, **proof,
|
|
184
187
|
processing_seconds=round(time.monotonic() - processing_started, 3))
|
|
185
|
-
receipt.update("processing", state=state,
|
|
188
|
+
receipt.update("processing", state=state,
|
|
186
189
|
processing_seconds=round(time.monotonic() - processing_started, 3))
|
|
187
|
-
if state == "FAILED":
|
|
188
|
-
raise UploadError("processing_failed",
|
|
189
|
-
"Apple build upload failed processing; inspect the upload ID's provider errors")
|
|
190
190
|
if time.monotonic() >= deadline:
|
|
191
191
|
raise UploadError("processing_pending",
|
|
192
192
|
"Apple build processing is pending; reuse this upload ID on continuation")
|
package/docs/CONFIGURATION.md
CHANGED
|
@@ -62,7 +62,11 @@ Compose service must pass `BACKEND_PUBLIC_BASE_URL` into its container and use i
|
|
|
62
62
|
The same named repository inputs reach the feature-branch backend preview in `deploy.yml`.
|
|
63
63
|
|
|
64
64
|
The supported managed files are `deploy.yml`, `mobile-candidate.yml`,
|
|
65
|
-
`warm-android-cache.yml`, `deploy-web.yml` and `deploy-backend.yml`.
|
|
65
|
+
`apple-upload-recovery.yml`, `warm-android-cache.yml`, `deploy-web.yml` and `deploy-backend.yml`.
|
|
66
|
+
The installer adds the dispatch-only `apple-upload-recovery.yml` when the rendered mobile
|
|
67
|
+
deployment supports iOS; explicit Android-only deployment excludes it. It preserves recovery
|
|
68
|
+
workflow overrides and removes only an unmodified generated copy when iOS is no longer supported.
|
|
69
|
+
The installer no longer adds
|
|
66
70
|
`deploy-web.yml`, and adds `warm-android-cache.yml` only while candidate artifacts are enabled
|
|
67
71
|
and `mobile.platforms` is not `ios`;
|
|
68
72
|
it removes such a file that is still the unmodified generated one with no overrides,
|
package/package.json
CHANGED
package/remedies.json
CHANGED
|
@@ -2,8 +2,8 @@
|
|
|
2
2
|
{
|
|
3
3
|
"schema": "gowalk-cicd/apple-build-upload-failed.v2",
|
|
4
4
|
"code": "processing_pending",
|
|
5
|
-
"fixed_in": "1.0.
|
|
6
|
-
"remedy": "Keep the original IPA and apple-build-upload receipt.
|
|
5
|
+
"fixed_in": "1.0.184",
|
|
6
|
+
"remedy": "Keep the original IPA and apple-build-upload receipt. From 1.0.184 recovery stages unchanged bytes as app.ipa so punctuation in the export filename cannot block retention. A PROCESSING upload summary can complete only after fresh proof of its completed matching asset/checksum and unique valid eligible iOS build, exact app/bundle and marketing version. Future receipts carry local MD5 as well as SHA-256. Run reconcile_apple_upload.cjs only in an authorized Actions recovery job with the original deployment, run, attempt and retained receipt; it never uploads. Read its owner completion back, then rerun the whole original deploy workflow. Missing original bytes or checksum proof remain unverified; never invent the digest, re-upload another binary under the same number or hand-edit the owner."
|
|
7
7
|
},
|
|
8
8
|
{
|
|
9
9
|
"schema": "gowalk-cicd/mobile-release-refused.v1",
|
|
@@ -26,8 +26,8 @@
|
|
|
26
26
|
{
|
|
27
27
|
"schema": "gowalk-cicd/mobile-release-refused.v1",
|
|
28
28
|
"code": "mobile_release_upload_unconfirmed_read_back_retained_binary_before_retry",
|
|
29
|
-
"fixed_in": "1.0.
|
|
30
|
-
"remedy": "Preserve the original upload owner and artifacts. If
|
|
29
|
+
"fixed_in": "1.0.185",
|
|
30
|
+
"remedy": "Preserve the original upload owner and artifacts. If the original iOS deployment failed or was cancelled after Build and transfer iOS binary and Retain Apple verification handoff succeeded in order, dispatch the managed apple-upload-recovery.yml on the default branch with original producer-run, producer-attempt, deployment-id and handoff-artifact-id. It authenticates the original artifact and unchanged handoff, terminal original and latest run, owner, and fresh Apple upload/file/build/app evidence. A null optional file checksum is absent; present whole-file checksums remain strictly verified. It never rebuilds, uploads, replays metadata or rewrites the original handoff. Retain the original handoff and derived completion proof with distinct current recovery caller, then rerun the whole original deployment: admission reuses the uploaded receipt without another iOS binary. The old failure or cancellation remains historical evidence; follow the new attempt to completion. Missing or mismatched evidence remains refused."
|
|
31
31
|
},
|
|
32
32
|
{
|
|
33
33
|
"schema": "gowalk-cicd/mobile-release-refused.v1",
|
package/src/config/workflows.mjs
CHANGED
|
@@ -36,11 +36,11 @@ function render(source, normalized, file) {
|
|
|
36
36
|
}
|
|
37
37
|
|
|
38
38
|
const WARM = 'warm-android-cache.yml';
|
|
39
|
-
const
|
|
39
|
+
const RECOVERY = 'apple-upload-recovery.yml';
|
|
40
|
+
const MANAGED = ['deploy.yml', 'mobile-candidate.yml', 'deploy-web.yml', 'deploy-backend.yml', WARM, RECOVERY];
|
|
40
41
|
|
|
41
|
-
//
|
|
42
|
-
//
|
|
43
|
-
// Anything customized stays and keeps upgrading.
|
|
42
|
+
// Retire generated workflows only while their locked bytes are unchanged and they have no overrides.
|
|
43
|
+
// App-owned customizations stay and keep upgrading when their automatic use is disabled.
|
|
44
44
|
function retired(root, file, state) {
|
|
45
45
|
if (state.migrating || state.config.workflows[file]?.overrides?.length) return false;
|
|
46
46
|
const previous = load(root, `.github/workflows/${file}`);
|
|
@@ -87,8 +87,14 @@ export function planWorkflows(root, templateDir, files, version) {
|
|
|
87
87
|
if (!MANAGED.includes(file)) throw new MigrationRepair('unknown_managed_workflow', CONFIG);
|
|
88
88
|
}
|
|
89
89
|
const removed = selected.filter(file => file === 'deploy-web.yml' && retired(root, file, state));
|
|
90
|
-
const writes = selected.filter(file => !removed.includes(file) && file
|
|
90
|
+
const writes = selected.filter(file => !removed.includes(file) && ![WARM, RECOVERY].includes(file))
|
|
91
91
|
.map(file => upgradeWorkflow(root, file, templateDir, state));
|
|
92
|
+
const recovery = selected.includes('deploy.yml') && state.config.mobile?.platforms !== 'android';
|
|
93
|
+
if (recovery || (selected.includes(RECOVERY) && !retired(root, RECOVERY, state))) {
|
|
94
|
+
writes.push(upgradeWorkflow(root, RECOVERY, templateDir, state));
|
|
95
|
+
} else if (selected.includes(RECOVERY)) {
|
|
96
|
+
removed.push(RECOVERY);
|
|
97
|
+
}
|
|
92
98
|
// Only verified candidate keys can request trusted post-release dependency priming on either OS.
|
|
93
99
|
const candidates = selected.includes('mobile-candidate.yml')
|
|
94
100
|
&& state.config.mobile?.['candidate-artifacts'] !== false;
|
|
@@ -106,7 +112,7 @@ export function planWorkflows(root, templateDir, files, version) {
|
|
|
106
112
|
for (const [file, entry] of Object.entries(state.config.workflows)) {
|
|
107
113
|
for (const change of entry.overrides) assertReferences(change.value, file, change.path);
|
|
108
114
|
}
|
|
109
|
-
writes.push(...planValidation(root, [...selected, WARM]));
|
|
115
|
+
writes.push(...planValidation(root, [...selected, WARM, RECOVERY]));
|
|
110
116
|
state.lock.package = version;
|
|
111
117
|
const config = state.text && equal(readYaml(state.text, CONFIG).value, state.config)
|
|
112
118
|
? state.text
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
name: Recover Apple upload
|
|
2
|
+
on:
|
|
3
|
+
workflow_dispatch:
|
|
4
|
+
inputs:
|
|
5
|
+
producer-run:
|
|
6
|
+
description: Original Mobile Deploy run ID
|
|
7
|
+
required: true
|
|
8
|
+
type: string
|
|
9
|
+
producer-attempt:
|
|
10
|
+
description: Original Mobile Deploy run attempt
|
|
11
|
+
required: true
|
|
12
|
+
type: string
|
|
13
|
+
deployment-id:
|
|
14
|
+
description: Original iOS release deployment ID
|
|
15
|
+
required: true
|
|
16
|
+
type: string
|
|
17
|
+
handoff-artifact-id:
|
|
18
|
+
description: Original Apple verification handoff artifact ID
|
|
19
|
+
required: true
|
|
20
|
+
type: string
|
|
21
|
+
|
|
22
|
+
permissions:
|
|
23
|
+
contents: read
|
|
24
|
+
actions: read
|
|
25
|
+
deployments: write
|
|
26
|
+
|
|
27
|
+
# Serialize the original owner update with ordinary Mobile Deploy; recovery never uploads.
|
|
28
|
+
concurrency:
|
|
29
|
+
group: mobile-deploy-${{ github.ref }}
|
|
30
|
+
cancel-in-progress: false
|
|
31
|
+
|
|
32
|
+
jobs:
|
|
33
|
+
recover:
|
|
34
|
+
name: Verify the original Apple upload
|
|
35
|
+
if: >-
|
|
36
|
+
${{ github.event_name == 'workflow_dispatch'
|
|
37
|
+
&& github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}
|
|
38
|
+
runs-on: ubuntu-24.04
|
|
39
|
+
timeout-minutes: 20
|
|
40
|
+
env:
|
|
41
|
+
GITHUB_TOKEN: ${{ github.token }}
|
|
42
|
+
APPLE_RECOVERY_PRODUCER_RUN: ${{ inputs.producer-run }}
|
|
43
|
+
APPLE_RECOVERY_PRODUCER_ATTEMPT: ${{ inputs.producer-attempt }}
|
|
44
|
+
APPLE_RECOVERY_DEPLOYMENT_ID: ${{ inputs.deployment-id }}
|
|
45
|
+
APPLE_RECOVERY_HANDOFF_ARTIFACT_ID: ${{ inputs.handoff-artifact-id }}
|
|
46
|
+
steps:
|
|
47
|
+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
|
|
48
|
+
- name: Validate original upload identifiers
|
|
49
|
+
shell: bash
|
|
50
|
+
run: |
|
|
51
|
+
for value in "$APPLE_RECOVERY_PRODUCER_RUN" "$APPLE_RECOVERY_PRODUCER_ATTEMPT" \
|
|
52
|
+
"$APPLE_RECOVERY_DEPLOYMENT_ID" "$APPLE_RECOVERY_HANDOFF_ARTIFACT_ID"; do
|
|
53
|
+
if [[ ! "$value" =~ ^[1-9][0-9]*$ ]]; then
|
|
54
|
+
echo "::error title=apple_upload_recovery_failed::invalid_recovery_identifiers"
|
|
55
|
+
exit 1
|
|
56
|
+
fi
|
|
57
|
+
done
|
|
58
|
+
- name: Prepare Apple API access
|
|
59
|
+
env:
|
|
60
|
+
CI_ASC_KEY_P8: ${{ secrets.ASC_KEY_P8 }}
|
|
61
|
+
CI_ASC_KEY_ID: ${{ secrets.ASC_KEY_ID }}
|
|
62
|
+
CI_ASC_ISSUER_ID: ${{ secrets.ASC_ISSUER_ID }}
|
|
63
|
+
run: |
|
|
64
|
+
umask 077
|
|
65
|
+
mkdir -p creds
|
|
66
|
+
if [ -n "${CI_ASC_KEY_P8:-}" ]; then
|
|
67
|
+
: "${CI_ASC_KEY_ID:?ASC_KEY_ID secret is required with ASC_KEY_P8}"
|
|
68
|
+
: "${CI_ASC_ISSUER_ID:?ASC_ISSUER_ID secret is required with ASC_KEY_P8}"
|
|
69
|
+
printf '%s' "$CI_ASC_KEY_P8" > "creds/AuthKey_${CI_ASC_KEY_ID}_Issuer_${CI_ASC_ISSUER_ID}.p8"
|
|
70
|
+
fi
|
|
71
|
+
python3 -m pip install --quiet --break-system-packages PyJWT cryptography requests
|
|
72
|
+
PYTHONPATH=.github/actions/swift-app/scripts python3 - <<'PY'
|
|
73
|
+
import os
|
|
74
|
+
from pathlib import Path
|
|
75
|
+
from read_config import emit_credentials
|
|
76
|
+
emit_credentials(Path(os.environ['GITHUB_ENV']), Path(os.environ['GITHUB_WORKSPACE']))
|
|
77
|
+
PY
|
|
78
|
+
- name: Verify the original upload and reconcile its owner
|
|
79
|
+
id: recover
|
|
80
|
+
run: |
|
|
81
|
+
node .github/actions/swift-app/scripts/apple_upload_recovery.cjs \
|
|
82
|
+
"$APPLE_RECOVERY_PRODUCER_RUN" "$APPLE_RECOVERY_PRODUCER_ATTEMPT" \
|
|
83
|
+
"$APPLE_RECOVERY_DEPLOYMENT_ID" "$APPLE_RECOVERY_HANDOFF_ARTIFACT_ID"
|
|
84
|
+
- name: Retain verified Apple recovery proof
|
|
85
|
+
if: ${{ steps.recover.outputs.verified == 'true' }}
|
|
86
|
+
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
|
|
87
|
+
env:
|
|
88
|
+
NODE_OPTIONS: --require "${{ github.workspace }}/.github/actions/android-app/scripts/artifact_env.cjs"
|
|
89
|
+
with:
|
|
90
|
+
name: apple-recovered-${{ github.run_id }}-${{ github.run_attempt }}
|
|
91
|
+
path: |
|
|
92
|
+
${{ runner.temp }}/apple-upload-recovery/apple-verification.json
|
|
93
|
+
${{ runner.temp }}/apple-upload-recovery/apple-verification.json.verified.json
|
|
94
|
+
${{ runner.temp }}/apple-upload-recovery/apple-verification.json.upload-receipt.json
|
|
95
|
+
if-no-files-found: error
|
|
96
|
+
retention-days: 7
|
|
Binary file
|