gowalk-cicd 1.0.60 → 1.0.62

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CLAUDE.md CHANGED
@@ -108,13 +108,24 @@ node /path/to/gowalk-cicd/bin/cli.mjs
108
108
  Python clients enforce explicit proxies; Play upload steps set the proxy for the
109
109
  pinned Google client. Missing configuration must stop before a provider call.
110
110
  Apple IPA uploads use `action/scripts/upload_build.py`, not native upload tools.
111
- It retains provider IDs/checksums and verifies matching IPA identity before resume.
111
+ It retains provider IDs/local SHA-256 and verifies matching IPA identity before resume.
112
+ Finalization sends `uploaded:true` as Apple's documented BuildUpload flow does; the
113
+ IPA endpoint rejected the optional SHA_256 checksum attribute despite the generic
114
+ schema advertising it. Completed-file reuse requires the content-derived filename
115
+ and size; any checksum returned by Apple is verified against the local file.
112
116
  - Signing material should use encrypted Actions secrets. `deploy.yml`
113
117
  materializes `ASC_KEY_P8` + its ID/issuer, the Apple distribution identity
114
118
  (`IOS_DISTRIBUTION_P12_BASE64`, `IOS_DISTRIBUTION_CERT_META_BASE64`,
115
119
  `IOS_DISTRIBUTION_CERT_REGISTRY_BASE64`), and the Android keystore, properties
116
120
  and Play service account into an ephemeral runner checkout.
117
121
  Tracked `creds/` files remain a backward-compatible fallback.
122
+ - Never revoke a distribution certificate automatically, including legacy caches.
123
+ `certificate-cap-policy` accepts only `fail`; a full cap requires registry reconciliation.
124
+ - Keep preinstalled Android SDK/NDKs. Every Gradle test/build and SDK download runs
125
+ under `play_store_proxy.py --`, whose owned loopback relay gives Java an explicit
126
+ authenticated account exit without credentials in JVM options. HTTPS is tunneled
127
+ without TLS interception; no upstream failure may fall back to the destination.
128
+ The wrapper disables Gradle daemons and closes its relay when the command ends.
118
129
  - Backend runtime credentials use the single encrypted `BACKEND_RUNTIME_ENV`
119
130
  secret. The action writes it to host-side `.runtime.env` with mode 0600 and
120
131
  supplies it to Compose after the generated `.env`; never print its content.
@@ -168,7 +179,7 @@ node /path/to/gowalk-cicd/bin/cli.mjs
168
179
  can be raised to cover a release's life.
169
180
  - Firebase Crashlytics traffic follows the Google account proxy. Android builds and
170
181
  the pinned Firebase CLI receive the explicit proxy environment; Crashlytics Gradle
171
- 2.9.2+ and the pinned buildtools support it (generic JVM networking does not).
182
+ 2.9.2+ and the pinned buildtools support it; generic JVM networking uses the relay.
172
183
  Native iOS upload-symbols uses NSURLSession, so it is never executed. Before
173
184
  archiving, dedicated Crashlytics upload phases in the ephemeral Xcode projects
174
185
  are deferred; mixed/unidentified upload phases refuse the archive for repair.
package/README.md CHANGED
@@ -327,12 +327,19 @@ zip I/O error: No space left on device
327
327
  ```
328
328
 
329
329
  The Android job removes the preinstalled toolchains a Flutter build never uses
330
- (.NET, the Android NDK, GHC, PowerShell, Swift, Chromium) and prunes Docker
330
+ (.NET, GHC, PowerShell, Swift, Chromium) and prunes Docker
331
331
  images, reclaiming roughly 25 GB in a few seconds. It prints `df -h /` before
332
332
  and after. Linux only; skipped in Bitrise mode. npm survives on purpose: the
333
333
  [Crashlytics symbol upload](#crashlytics-symbol-delivery-firebase_app_id) runs
334
334
  the Firebase CLI through `npx`.
335
335
 
336
+ The preinstalled Android SDK and NDKs are retained. Gradle and SDK-manager Java
337
+ networking runs through an owned loopback relay to `GOOGLE_STORE_PROXY_URL`.
338
+ The relay authenticates to that exit without putting credentials in JVM options;
339
+ HTTPS remains an end-to-end TLS tunnel. An unavailable proxy fails the request,
340
+ and the relay shuts down with its child command. Gradle daemons are disabled for
341
+ these commands so they cannot retain an expired relay port.
342
+
336
343
  ### Private git dependencies (Flutter)
337
344
 
338
345
  A Flutter app can depend on private git packages:
@@ -441,7 +448,8 @@ identity and create per-app provisioning profiles, but it never creates,
441
448
  replaces, or revokes a registry-managed distribution certificate. Missing,
442
449
  partially written, corrupt, expired, Apple-revoked, or resource-ID/P12-mismatched
443
450
  managed material aborts with a reconciliation error. Legacy repos without the
444
- marker keep the historical cache lifecycle.
451
+ marker may create a certificate in an available slot, but never revoke another
452
+ identity. A full certificate cap stops for account-registry reconciliation.
445
453
 
446
454
  ## How it works
447
455
 
@@ -773,7 +781,7 @@ common ones:
773
781
  | `bundle-id` | Override the auto-detected bundle identifier |
774
782
  | `team-id` | Override the auto-detected team ID |
775
783
  | `app-store-apple-id` | Numeric ASC app ID (override auto-lookup) |
776
- | `certificate-cap-policy` | Full-cap behavior: `fail` preserves identities; default `revoke-oldest` rotates. |
784
+ | `certificate-cap-policy` | Only `fail` is accepted (the default); existing identities are always preserved. |
777
785
  | `run-tests` | `false` to skip the simulator test stage |
778
786
  | `uses-non-exempt-encryption` | Value for `ITSAppUsesNonExemptEncryption` |
779
787
  | `archive` | `false` to build-only (PR runs without secrets) |
@@ -793,8 +801,7 @@ inside GitHub Actions.
793
801
  Once the app exists, all subsequent builds and uploads are fully automated via
794
802
  the ASC API key.
795
803
 
796
- For accounts where automatic certificate revocation is not acceptable, pass
797
- `certificate-cap-policy: 'fail'`. The action may add a certificate when Apple has a free
804
+ Automatic certificate revocation is disabled for every account. The action may add a certificate when Apple has a free
798
805
  slot, but a full-cap response aborts without listing or revoking existing identities. If
799
806
  signing preparation fails after creating a certificate, the default-branch workflow first
800
807
  commits any completed cache files, then re-raises the failure so the private key is not lost.
@@ -955,3 +962,16 @@ even that first auto-bootstrap.
955
962
  ## License
956
963
 
957
964
  MIT
965
+
966
+ ### Recovering an Apple REST upload failure
967
+
968
+ The uploader retains the local SHA-256, version and provider upload/file IDs. It commits
969
+ completed part transfers with `uploaded: true`, following Apple's
970
+ [BuildUpload walkthrough](https://developer.apple.com/videos/play/wwdc2025/324/).
971
+ The IPA endpoint rejected an optional `sourceFileChecksums` SHA_256 object even though
972
+ the generic schema lists that algorithm; the uploader no longer sends that attribute.
973
+ A completed file is reused only when its content-derived filename and size match, and
974
+ any returned file checksum is verified. A failed upload emits the typed
975
+ `apple_build_upload_failed` error annotation with schema
976
+ `gowalk-cicd/apple-build-upload-failed.v1`; retain the exact IPA/receipt and read back
977
+ the provider state before retrying. All requests and parts still use the Apple proxy.
@@ -1 +1 @@
1
- 1.0.60
1
+ 1.0.62
package/action/action.yml CHANGED
@@ -53,10 +53,10 @@ inputs:
53
53
  certificate-cap-policy:
54
54
  description: >
55
55
  Behavior when Apple reports the two-certificate Distribution cap.
56
- 'revoke-oldest' preserves historical behavior; 'fail' aborts without
57
- listing or revoking any existing certificate.
56
+ Only 'fail' is supported: abort without revoking any existing certificate.
57
+ Reconcile the account registry when no live signing identity is available.
58
58
  required: false
59
- default: "revoke-oldest"
59
+ default: "fail"
60
60
  persist-signing-cache:
61
61
  description: >
62
62
  Commit refreshed signing files back to the default branch. Set to
@@ -3,7 +3,7 @@
3
3
  Cert-creation primitives for the iOS native TestFlight action.
4
4
 
5
5
  Owns the cryptographic legwork (RSA key + CSR), Apple's per-team cert
6
- cap policy (fail closed or revoke OLDEST on 409), and PKCS12 serialisation. Split
6
+ cap refusal (never automatic revocation), and PKCS12 serialisation. Split
7
7
  from ``prepare_signing.py`` so the orchestrator there stays focused on
8
8
  the load-or-regen control flow and the file count stays under the
9
9
  project's per-file limits.
@@ -16,35 +16,20 @@ from __future__ import annotations
16
16
 
17
17
  import base64
18
18
  import os
19
- import time
20
19
 
21
- from asc_common import get_json, request
20
+ from asc_common import request
22
21
  from cryptography import x509
23
22
  from cryptography.hazmat.primitives import hashes, serialization
24
23
  from cryptography.hazmat.primitives.asymmetric import rsa
25
24
  from cryptography.hazmat.primitives.serialization import pkcs12
26
25
  from cryptography.x509.oid import NameOID
27
26
 
28
- # Apple's cert revoke -> create pipeline is eventually consistent: a
29
- # DELETE on the per-team cap-blocking cert sometimes still shows up as
30
- # "active" to a follow-up POST for a second or two, returning a fresh
31
- # 409 even though we just freed a slot. Sleep then retry-with-backoff
32
- # rather than failing the whole CI run on a known-transient race.
33
- CERT_REVOKE_PROPAGATION_DELAY_SEC = float(
34
- os.getenv("CERT_REVOKE_PROPAGATION_DELAY_SEC", "2.0")
35
- )
36
- CERT_POST_RETRIES_AFTER_REVOKE = int(
37
- os.getenv("CERT_POST_RETRIES_AFTER_REVOKE", "2")
38
- )
39
- CERTIFICATE_CAP_POLICIES = frozenset({"revoke-oldest", "fail"})
40
-
41
27
 
42
28
  def certificate_cap_policy() -> str:
43
- """Return the configured full-cap behavior before any provider call."""
44
- policy = os.getenv("CERTIFICATE_CAP_POLICY", "revoke-oldest").strip().lower()
45
- if policy not in CERTIFICATE_CAP_POLICIES:
46
- allowed = ", ".join(sorted(CERTIFICATE_CAP_POLICIES))
47
- raise SystemExit(f"invalid CERTIFICATE_CAP_POLICY {policy!r}; use {allowed}")
29
+ """Refuse destructive legacy policy before any provider call."""
30
+ policy = os.getenv("CERTIFICATE_CAP_POLICY", "fail").strip().lower()
31
+ if policy != "fail":
32
+ raise SystemExit("invalid CERTIFICATE_CAP_POLICY; use fail. Automatic certificate revocation is disabled")
48
33
  return policy
49
34
 
50
35
 
@@ -75,79 +60,9 @@ def generate_key_and_csr() -> tuple[rsa.RSAPrivateKey, bytes]:
75
60
  return private_key, csr_payload
76
61
 
77
62
 
78
- def oldest_distribution_cert_id(token: str) -> str | None:
79
- """Return the DISTRIBUTION cert with the EARLIEST expirationDate.
80
-
81
- Apple caps each team at 2 distribution certs; when CI hits the cap
82
- we must revoke one. Picking the OLDEST is the safest choice — the
83
- NEWEST cert signed the most recent build that may still be in ASC
84
- processing, and revoking that mid-processing produces ITMS-90035
85
- ("signed with an ad-hoc certificate, not a distribution
86
- certificate") on the prior run. Older certs have long since cleared
87
- processing.
88
- """
89
- data = get_json(
90
- "/certificates",
91
- token,
92
- params={
93
- "limit": "200",
94
- "sort": "-id",
95
- "filter[certificateType]": "DISTRIBUTION",
96
- },
97
- )
98
- oldest_id = None
99
- oldest_exp: str | None = None
100
- for cert in data.get("data", []):
101
- attrs = cert.get("attributes") or {}
102
- exp = attrs.get("expirationDate") or ""
103
- if not exp:
104
- continue
105
- if oldest_exp is None or exp < oldest_exp:
106
- oldest_exp = exp
107
- oldest_id = cert["id"]
108
- return oldest_id
109
-
110
-
111
- def _post_with_revoke_backoff(token: str, body: dict):
112
- """POST a cert create after a revoke, tolerating ASC propagation lag.
113
-
114
- Apple's DELETE -> POST pipeline is eventually consistent: a freshly
115
- revoked cert may still count against the per-team cap for a brief
116
- window, producing a spurious 409 on the immediate re-POST. Sleep
117
- once for ``CERT_REVOKE_PROPAGATION_DELAY_SEC`` then retry up to
118
- ``CERT_POST_RETRIES_AFTER_REVOKE`` times with exponential backoff.
119
- The final attempt drops ``allow_status`` so a real 409 surfaces as
120
- Apple's full body via ``request``'s SystemExit.
121
- """
122
- time.sleep(CERT_REVOKE_PROPAGATION_DELAY_SEC)
123
- delay = CERT_REVOKE_PROPAGATION_DELAY_SEC
124
- for attempt in range(CERT_POST_RETRIES_AFTER_REVOKE):
125
- resp = request(
126
- "POST", "/certificates", token, json_body=body, allow_status={409}
127
- )
128
- if resp.status_code != 409:
129
- return resp
130
- print(
131
- f"Post-revoke 409 on attempt {attempt + 1}/"
132
- f"{CERT_POST_RETRIES_AFTER_REVOKE}; "
133
- f"sleeping {delay}s before final retry"
134
- )
135
- time.sleep(delay)
136
- delay *= 2
137
- # Final attempt without allow_status — let Apple's body surface via
138
- # request()'s SystemExit if the cap is genuinely still hit.
139
- return request("POST", "/certificates", token, json_body=body)
140
-
141
-
142
63
  def create_distribution_cert(token: str, csr_b64: str) -> tuple[str, bytes]:
143
- """POST a CSR to ASC; return ``(cert_id, cert_der)``.
144
-
145
- On 409 (per-team cap of 2 hit) revoke the OLDEST existing cert
146
- (see :func:`oldest_distribution_cert_id` for why) and retry with
147
- backoff to absorb Apple's revoke -> create propagation lag (see
148
- :func:`_post_with_revoke_backoff`).
149
- """
150
- cap_policy = certificate_cap_policy()
64
+ """POST one CSR; a full team cap preserves every existing certificate."""
65
+ certificate_cap_policy()
151
66
  body = {
152
67
  "data": {
153
68
  "type": "certificates",
@@ -161,24 +76,10 @@ def create_distribution_cert(token: str, csr_b64: str) -> tuple[str, bytes]:
161
76
  "POST", "/certificates", token, json_body=body, allow_status={409}
162
77
  )
163
78
  if resp.status_code == 409:
164
- if cap_policy == "fail":
165
- raise SystemExit(
166
- "distribution certificate cap reached; policy=fail preserves all "
167
- "existing certificates and refuses automatic revocation"
168
- )
169
- # Revoke the OLDEST cert, NOT the newest. The newest cert signed
170
- # the previous build that may still be in ASC processing — revoking
171
- # it during processing yields ITMS-90035 on the prior build.
172
- # See oldest_distribution_cert_id() for the full rationale.
173
- print("Distribution cert cap hit; revoking oldest existing cert")
174
- target = oldest_distribution_cert_id(token)
175
- if not target:
176
- raise SystemExit(
177
- "409 from cert create but no existing DISTRIBUTION cert "
178
- "found to revoke"
179
- )
180
- request("DELETE", f"/certificates/{target}", token)
181
- resp = _post_with_revoke_backoff(token, body)
79
+ raise SystemExit(
80
+ "distribution certificate cap reached; policy=fail preserves all "
81
+ "existing certificates and refuses automatic revocation; reconcile the account registry"
82
+ )
182
83
  data = resp.json()["data"]
183
84
  cert_id = data["id"]
184
85
  cert_der = base64.b64decode(data["attributes"]["certificateContent"])
@@ -220,7 +220,7 @@ def verify_cert_alive(token: str, cert_id: str,
220
220
  """Return True iff GET /certificates/{cert_id} returns 200.
221
221
 
222
222
  A 404 means Apple revoked it (manually or via the cap-rotation done
223
- by ``oldest_distribution_cert_id``); any other non-200 is treated
223
+ by another signing client); any other non-200 is treated
224
224
  conservatively as not-alive so we regenerate.
225
225
  """
226
226
  resp = request(
@@ -28,27 +28,25 @@ class CertificateCapPolicyTests(unittest.TestCase):
28
28
  response = mock.MagicMock(status_code=409)
29
29
  with mock.patch.dict(os.environ, {"CERTIFICATE_CAP_POLICY": "fail"}, clear=True):
30
30
  with mock.patch.object(cert_factory, "request", return_value=response) as request:
31
- with mock.patch.object(cert_factory, "oldest_distribution_cert_id") as oldest:
32
- with self.assertRaisesRegex(SystemExit, "refuses automatic revocation"):
33
- cert_factory.create_distribution_cert("token", "csr")
31
+ with self.assertRaisesRegex(SystemExit, "refuses automatic revocation"):
32
+ cert_factory.create_distribution_cert("token", "csr")
34
33
  request.assert_called_once()
35
- oldest.assert_not_called()
34
+ self.assertEqual(request.call_args.args[:2], ("POST", "/certificates"))
36
35
 
37
- def test_default_policy_keeps_backward_compatible_rotation(self):
36
+ def test_default_policy_never_revokes_or_retries_a_full_cap(self):
38
37
  capped = mock.MagicMock(status_code=409)
39
- created = mock.MagicMock()
40
- created.json.return_value = {
41
- "data": {"id": "NEW", "attributes": {"certificateContent": "Y2VydA=="}}
42
- }
43
38
  with mock.patch.dict(os.environ, {}, clear=True):
44
- with mock.patch.object(
45
- cert_factory, "request", side_effect=[capped, mock.MagicMock()]
46
- ) as request:
47
- with mock.patch.object(cert_factory, "oldest_distribution_cert_id", return_value="OLD"):
48
- with mock.patch.object(cert_factory, "_post_with_revoke_backoff", return_value=created):
49
- cert_id, cert_der = cert_factory.create_distribution_cert("token", "csr")
50
- self.assertEqual((cert_id, cert_der), ("NEW", b"cert"))
51
- request.assert_any_call("DELETE", "/certificates/OLD", "token")
39
+ with mock.patch.object(cert_factory, "request", return_value=capped) as request:
40
+ with self.assertRaisesRegex(SystemExit, "refuses automatic revocation"):
41
+ cert_factory.create_distribution_cert("token", "csr")
42
+ request.assert_called_once()
43
+
44
+ def test_historical_destructive_setting_is_refused_before_contacting_apple(self):
45
+ with mock.patch.dict(os.environ, {"CERTIFICATE_CAP_POLICY": "revoke-oldest"}, clear=True):
46
+ with mock.patch.object(cert_factory, "request") as request:
47
+ with self.assertRaisesRegex(SystemExit, "Automatic certificate revocation is disabled"):
48
+ cert_factory.create_distribution_cert("token", "csr")
49
+ request.assert_not_called()
52
50
 
53
51
 
54
52
  class CertificateCapActionWiringTests(unittest.TestCase):
@@ -56,7 +54,7 @@ class CertificateCapActionWiringTests(unittest.TestCase):
56
54
  source = ACTION_YAML.read_text(encoding="utf-8")
57
55
  self.assertRegex(
58
56
  source,
59
- r"(?m)^ certificate-cap-policy:\n(?: .*\n)*? default: \"revoke-oldest\"$",
57
+ r"(?m)^ certificate-cap-policy:\n(?: .*\n)*? default: \"fail\"$",
60
58
  )
61
59
  self.assertIn("CERTIFICATE_CAP_POLICY: ${{ inputs.certificate-cap-policy }}", source)
62
60
  self.assertIn("SIGNING_PREP_EXIT=$signing_exit", source)
@@ -62,7 +62,7 @@ class BuildUploadTests(unittest.TestCase):
62
62
  api.assert_not_called()
63
63
  transfer.assert_not_called()
64
64
 
65
- def test_parts_are_joined_before_committing_the_checksum(self):
65
+ def test_parts_are_joined_before_committing_the_uploaded_file(self):
66
66
  file = {"id": "file", "attributes": {"uploadOperations": [{"offset": 0, "length": 2},
67
67
  {"offset": 2, "length": 3}]}}
68
68
  done = []
@@ -71,8 +71,7 @@ class BuildUploadTests(unittest.TestCase):
71
71
  upload_build.transfer_all(Path("unused"), file, {"fileSize": 5, "sha256": "digest"})
72
72
  self.assertEqual(sorted(done), [0, 2])
73
73
  body = api.call_args.args[2]["data"]["attributes"]
74
- self.assertEqual(body, {"uploaded": True, "sourceFileChecksums": {
75
- "file": {"hash": "digest", "algorithm": "SHA_256"}}})
74
+ self.assertEqual(body, {"uploaded": True})
76
75
 
77
76
  def test_failed_part_never_commits_the_file(self):
78
77
  file = {"id": "file", "attributes": {"uploadOperations": [{"offset": 0, "length": 5}]}}
@@ -0,0 +1,74 @@
1
+ """Completed-file recovery keeps byte identity without inventing optional provider attributes."""
2
+ import base64
3
+ from contextlib import redirect_stdout
4
+ import hashlib
5
+ import io
6
+ from pathlib import Path
7
+ import sys
8
+ import tempfile
9
+ import unittest
10
+ from unittest import mock
11
+
12
+ sys.path.insert(0, str(Path(__file__).resolve().parent))
13
+ import upload_build
14
+
15
+
16
+ class CompletedUploadTests(unittest.TestCase):
17
+ def test_provider_exit_emits_typed_failure_without_exception_content(self):
18
+ output = io.StringIO()
19
+ with tempfile.TemporaryDirectory() as temporary, redirect_stdout(output), \
20
+ mock.patch.object(sys, "argv", ["upload_build.py", "app.ipa", "--report", temporary + "/receipt.json"]), \
21
+ mock.patch.object(upload_build, "deliver", side_effect=SystemExit("private-provider-detail")):
22
+ with self.assertRaisesRegex(SystemExit, "retain its receipt"):
23
+ upload_build.main()
24
+ self.assertIn("::error title=apple_build_upload_failed::", output.getvalue())
25
+ self.assertNotIn("private-provider-detail", output.getvalue())
26
+
27
+ def test_provider_completion_without_optional_checksum_reuses_exact_file(self):
28
+ info = {"fileName": "source-sha256.ipa", "fileSize": 5}
29
+ file = {"attributes": {**info, "assetDeliveryState": {"state": "COMPLETE"}}}
30
+ with mock.patch.object(upload_build, "api") as api, mock.patch.object(upload_build, "transfer") as transfer:
31
+ upload_build.transfer_all(Path("not-read"), file, info)
32
+ api.assert_not_called()
33
+ transfer.assert_not_called()
34
+ file["attributes"]["fileName"] = "another-source.ipa"
35
+ with self.assertRaisesRegex(ValueError, "identity"):
36
+ upload_build.transfer_all(Path("not-read"), file, info)
37
+
38
+ def test_present_provider_checksums_are_verified_against_actual_bytes(self):
39
+ with tempfile.TemporaryDirectory() as temporary:
40
+ ipa = Path(temporary) / "app.ipa"
41
+ ipa.write_bytes(b"the uploaded file")
42
+ info = {"fileName": "source-sha256.ipa", "fileSize": ipa.stat().st_size}
43
+ for algorithm, digest in [("SHA_256", hashlib.sha256(ipa.read_bytes())),
44
+ ("MD5", hashlib.md5(ipa.read_bytes()))]:
45
+ for value in [digest.hexdigest(), base64.b64encode(digest.digest()).decode()]:
46
+ with self.subTest(algorithm=algorithm, encoding=value):
47
+ attributes = {**info, "assetDeliveryState": {"state": "COMPLETE"},
48
+ "sourceFileChecksums": {"file": {"hash": value, "algorithm": algorithm}}}
49
+ upload_build.transfer_all(ipa, {"attributes": attributes}, info)
50
+ attributes["sourceFileChecksums"]["file"]["hash"] = "different-bytes"
51
+ with self.assertRaisesRegex(ValueError, "checksum"):
52
+ upload_build.transfer_all(ipa, {"attributes": attributes}, info)
53
+
54
+ def test_commit_uses_documented_uploaded_attribute_after_all_bytes_finish(self):
55
+ # Apple's WWDC25 BuildUpload sample sends uploaded=true. The observed IPA
56
+ # endpoint rejected the optional SHA_256 sourceFileChecksums object with409.
57
+ parts = [{"offset": 0, "length": 2}, {"offset": 2, "length": 3}]
58
+ done = []
59
+ def provider(method, path, body):
60
+ self.assertEqual(sorted(done), [0, 2])
61
+ attributes = body["data"]["attributes"]
62
+ if "sourceFileChecksums" in attributes:
63
+ raise SystemExit("409 ENTITY_ERROR.ATTRIBUTE.INVALID")
64
+ self.assertTrue(attributes["uploaded"])
65
+ return {"data": {"id": "existing-file"}}
66
+ with mock.patch.object(upload_build, "api", side_effect=provider) as api, \
67
+ mock.patch.object(upload_build, "transfer", side_effect=lambda _path, op: done.append(op["offset"])):
68
+ upload_build.transfer_all(Path("not-read"), {"id": "existing-file", "attributes": {
69
+ "uploadOperations": parts}}, {"fileSize": 5, "sha256": "local-receipt-digest"})
70
+ self.assertEqual(api.call_count, 1)
71
+
72
+
73
+ if __name__ == "__main__":
74
+ unittest.main()
@@ -15,6 +15,7 @@ import zipfile
15
15
 
16
16
  import asc_common
17
17
  import apple_store_proxy as store_proxy
18
+ import upload_checksums
18
19
 
19
20
 
20
21
  def api(method: str, path: str, body=None, params=None) -> dict:
@@ -96,9 +97,7 @@ def transfer_all(ipa: Path, file: dict, info: dict) -> None:
96
97
  attributes = file.get("attributes") or {}
97
98
  state = (attributes.get("assetDeliveryState") or {}).get("state")
98
99
  if state == "COMPLETE":
99
- expected = {"hash": info["sha256"], "algorithm": "SHA_256"}
100
- if (attributes.get("sourceFileChecksums") or {}).get("file") != expected:
101
- raise ValueError("Apple's completed file checksum does not match this IPA")
100
+ upload_checksums.completed(ipa, attributes, info)
102
101
  return
103
102
  operations = sorted(attributes.get("uploadOperations") or [], key=lambda row: row["offset"])
104
103
  offset = 0
@@ -113,8 +112,10 @@ def transfer_all(ipa: Path, file: dict, info: dict) -> None:
113
112
  for result in results:
114
113
  result.result()
115
114
  api("PATCH", f"/buildUploadFiles/{file['id']}", {"data": {
116
- "type": "buildUploadFiles", "id": file["id"], "attributes": {"uploaded": True,
117
- "sourceFileChecksums": {"file": {"hash": info["sha256"], "algorithm": "SHA_256"}}},
115
+ # Apple's build-upload contract commits with uploaded=true. Its generic
116
+ # checksum enum includes SHA_256, but IPA finalization rejected that
117
+ # optional attribute with ENTITY_ERROR.ATTRIBUTE.INVALID in production.
118
+ "type": "buildUploadFiles", "id": file["id"], "attributes": {"uploaded": True},
118
119
  }})
119
120
 
120
121
 
@@ -154,7 +155,11 @@ def main() -> None:
154
155
  args = parser.parse_args()
155
156
  try:
156
157
  receipt = deliver(args.ipa, args.report, args.timeout)
157
- except (OSError, ValueError, KeyError, RuntimeError):
158
+ except (OSError, ValueError, KeyError, RuntimeError, SystemExit):
159
+ print('::error title=apple_build_upload_failed::' + json.dumps({
160
+ "schema": "gowalk-cicd/apple-build-upload-failed.v1",
161
+ "receipt_available": args.report.is_file(),
162
+ "action": "retain the IPA and receipt; read provider upload state before retrying"}))
158
163
  message = "Apple REST upload did not complete; retain its receipt and resolve the provider state"
159
164
  raise SystemExit(message) from None
160
165
  print(json.dumps(receipt))
@@ -0,0 +1,24 @@
1
+ """Check completed upload identity without requiring Apple's optional checksum attribute."""
2
+ from __future__ import annotations
3
+
4
+ import base64
5
+ import hashlib
6
+ from pathlib import Path
7
+
8
+
9
+ def completed(ipa: Path, attributes: dict, info: dict) -> None:
10
+ if any(attributes.get(key) != info[key] for key in ("fileName", "fileSize")):
11
+ raise ValueError("Apple's completed file identity does not match this IPA")
12
+ checksum = (attributes.get("sourceFileChecksums") or {}).get("file")
13
+ if checksum is None:
14
+ # The name is the local SHA-256 plus .ipa; upload_file verified this same
15
+ # provider-owned file identity before any transfer or completed-file reuse.
16
+ return
17
+ algorithm = {"SHA_256": "sha256", "MD5": "md5"}.get(checksum.get("algorithm"))
18
+ if algorithm is None:
19
+ raise ValueError("Apple returned an unsupported completed-file checksum algorithm")
20
+ with ipa.open("rb") as stream:
21
+ digest = hashlib.file_digest(stream, algorithm)
22
+ expected = {digest.hexdigest(), digest.hexdigest().upper(), base64.b64encode(digest.digest()).decode()}
23
+ if checksum.get("hash") not in expected:
24
+ raise ValueError("Apple's completed file checksum does not match this IPA")
@@ -1 +1 @@
1
- 1.0.60
1
+ 1.0.62
@@ -222,7 +222,8 @@ runs:
222
222
  working-directory: ${{ steps.config.outputs.gradle_root || '.' }}
223
223
  run: |
224
224
  chmod +x ./gradlew
225
- ./gradlew test --console=plain --stacktrace
225
+ python3 "${{ github.action_path }}/scripts/play_store_proxy.py" -- \
226
+ ./gradlew test --console=plain --stacktrace
226
227
 
227
228
  # ANDROID_BUILD_NUMBER is already in the environment: resolve_android.py
228
229
  # exported it through GITHUB_ENV. ANDROID_BUILD_NAME is only set when the
@@ -0,0 +1,119 @@
1
+ """An owned loopback relay for JVMs that cannot authenticate to the assigned HTTP proxy.
2
+
3
+ Only the configured upstream is ever dialled. HTTPS remains an opaque CONNECT
4
+ tunnel; Java still verifies the provider's TLS certificate end to end.
5
+ """
6
+ from __future__ import annotations
7
+
8
+ import asyncio
9
+ import base64
10
+ from contextlib import contextmanager
11
+ import ssl
12
+ import threading
13
+ from urllib.parse import unquote, urlsplit
14
+
15
+
16
+ class Relay:
17
+ def __init__(self, proxy: str):
18
+ self.proxy = urlsplit(proxy)
19
+ self.loop = asyncio.new_event_loop()
20
+ self.ready = threading.Event()
21
+ self.thread = threading.Thread(target=self._serve, daemon=True)
22
+ self.server = None
23
+ self.port = None
24
+
25
+ def _serve(self):
26
+ asyncio.set_event_loop(self.loop)
27
+ try:
28
+ self.server = self.loop.run_until_complete(
29
+ asyncio.start_server(self._handle, "127.0.0.1", 0, limit=65536))
30
+ self.port = self.server.sockets[0].getsockname()[1]
31
+ self.ready.set()
32
+ self.loop.run_forever()
33
+ finally:
34
+ self.ready.set()
35
+ if self.server:
36
+ self.server.close()
37
+ self.loop.run_until_complete(self.server.wait_closed())
38
+ tasks = asyncio.all_tasks(self.loop)
39
+ for task in tasks:
40
+ task.cancel()
41
+ self.loop.run_until_complete(asyncio.gather(*tasks, return_exceptions=True))
42
+ self.loop.close()
43
+
44
+ async def _upstream(self, request: bytes):
45
+ secure = ssl.create_default_context() if self.proxy.scheme == "https" else None
46
+ reader, writer = await asyncio.wait_for(asyncio.open_connection(
47
+ self.proxy.hostname, self.proxy.port, ssl=secure), timeout=30)
48
+ lines = request.decode("iso-8859-1").split("\r\n")
49
+ lines = [line for line in lines if line and not line.lower().startswith("proxy-authorization:")]
50
+ if self.proxy.username is not None:
51
+ login = f"{unquote(self.proxy.username)}:{unquote(self.proxy.password or '')}"
52
+ encoded = base64.b64encode(login.encode()).decode("ascii")
53
+ lines.append("Proxy-Authorization: Basic " + encoded)
54
+ writer.write(("\r\n".join(lines) + "\r\n\r\n").encode("iso-8859-1"))
55
+ await writer.drain()
56
+ return reader, writer
57
+
58
+ async def _pump(self, reader, writer):
59
+ while data := await asyncio.wait_for(reader.read(65536), timeout=120):
60
+ writer.write(data)
61
+ await writer.drain()
62
+
63
+ async def _handle(self, reader, writer):
64
+ upstream = None
65
+ pumps = []
66
+ try:
67
+ request = await asyncio.wait_for(reader.readuntil(b"\r\n\r\n"), timeout=15)
68
+ upstream_reader, upstream = await self._upstream(request)
69
+ if request.startswith(b"CONNECT "):
70
+ answer = await asyncio.wait_for(upstream_reader.readuntil(b"\r\n\r\n"), timeout=30)
71
+ if answer.split(b" ", 2)[1] != b"200":
72
+ raise ConnectionError("upstream tunnel refused")
73
+ writer.write(b"HTTP/1.1 200 Connection established\r\n\r\n")
74
+ await writer.drain()
75
+ pumps = [asyncio.create_task(self._pump(reader, upstream)),
76
+ asyncio.create_task(self._pump(upstream_reader, writer))]
77
+ await asyncio.wait(pumps, return_when=asyncio.FIRST_COMPLETED)
78
+ except (OSError, ValueError, IndexError, asyncio.TimeoutError,
79
+ asyncio.IncompleteReadError, asyncio.LimitOverrunError):
80
+ writer.write(b"HTTP/1.1 502 Proxy unavailable\r\nContent-Length: 0\r\nConnection: close\r\n\r\n")
81
+ finally:
82
+ for task in pumps:
83
+ task.cancel()
84
+ if pumps:
85
+ await asyncio.gather(*pumps, return_exceptions=True)
86
+ for connection in (writer, upstream):
87
+ if connection:
88
+ connection.close()
89
+ try:
90
+ await connection.wait_closed()
91
+ except OSError:
92
+ pass
93
+
94
+
95
+ @contextmanager
96
+ def running(proxy: str):
97
+ relay = Relay(proxy)
98
+ relay.thread.start()
99
+ try:
100
+ if not relay.ready.wait(10) or relay.port is None:
101
+ raise RuntimeError("Could not start the owned JVM proxy relay")
102
+ yield relay.port
103
+ finally:
104
+ if relay.loop.is_running():
105
+ relay.loop.call_soon_threadsafe(relay.loop.stop)
106
+ relay.thread.join(timeout=10)
107
+ if relay.thread.is_alive():
108
+ raise RuntimeError("Owned JVM proxy relay did not finish cleanup")
109
+
110
+
111
+ def environment(env: dict, port: int) -> dict:
112
+ options = (f"-Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort={port} "
113
+ f"-Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort={port} "
114
+ "-Dhttp.nonProxyHosts= -Dhttps.nonProxyHosts= -Djava.net.useSystemProxies=false")
115
+ # Java prints these options at startup; they contain no proxy credentials.
116
+ for name in ("JAVA_TOOL_OPTIONS", "JDK_JAVA_OPTIONS", "_JAVA_OPTIONS", "GRADLE_OPTS"):
117
+ env[name] = (env.get(name, "") + " " + options).strip()
118
+ env["GRADLE_OPTS"] += " -Dorg.gradle.daemon=false"
119
+ return env
@@ -53,8 +53,11 @@ def run(argv: list[str]) -> int:
53
53
  """Run ``argv`` with the explicit proxy environment; the parent process is untouched."""
54
54
  if not argv:
55
55
  raise SystemExit("usage: play_store_proxy.py -- <command …>")
56
+ from jvm_proxy_relay import environment as jvm_environment, running
57
+
56
58
  env = environment()
57
- return subprocess.run(argv, env=env, check=False).returncode
59
+ with running(proxy_url()) as port:
60
+ return subprocess.run(argv, env=jvm_environment(env, port), check=False).returncode
58
61
 
59
62
 
60
63
  def clear_github_env_bypass(path: str | None = None) -> list[str]:
@@ -0,0 +1,99 @@
1
+ """Real socket and Java proofs: authenticated upstream only, no target fallback."""
2
+ import base64
3
+ from contextlib import closing, contextmanager
4
+ import http.client
5
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
6
+ import os
7
+ from pathlib import Path
8
+ import shutil
9
+ import socket
10
+ import subprocess
11
+ import tempfile
12
+ import threading
13
+ import unittest
14
+
15
+ import jvm_proxy_relay as relay
16
+
17
+
18
+ @contextmanager
19
+ def upstream():
20
+ seen = []
21
+
22
+ class Proxy(BaseHTTPRequestHandler):
23
+ def do_GET(self):
24
+ seen.append((self.command, self.path, self.headers.get("Proxy-Authorization")))
25
+ self.send_response(200)
26
+ self.end_headers()
27
+ self.wfile.write(b"through pinned proxy")
28
+
29
+ def do_CONNECT(self):
30
+ seen.append((self.command, self.path, self.headers.get("Proxy-Authorization")))
31
+ self.send_response(200)
32
+ self.end_headers()
33
+ value = self.rfile.read(4)
34
+ self.wfile.write(value)
35
+
36
+ def log_message(self, *_):
37
+ pass
38
+
39
+ server = ThreadingHTTPServer(("127.0.0.1", 0), Proxy)
40
+ server.daemon_threads = True
41
+ thread = threading.Thread(target=server.serve_forever, daemon=True)
42
+ thread.start()
43
+ try:
44
+ yield f"http://fixture-user:fixture-pass@127.0.0.1:{server.server_port}", seen
45
+ finally:
46
+ server.shutdown()
47
+ server.server_close()
48
+ thread.join(timeout=5)
49
+
50
+
51
+ class JvmProxyTests(unittest.TestCase):
52
+ def test_http_and_connect_only_reach_the_authenticated_upstream_and_cleanup(self):
53
+ with upstream() as (proxy, seen):
54
+ with relay.running(proxy) as port:
55
+ with closing(http.client.HTTPConnection("127.0.0.1", port, timeout=5)) as client:
56
+ client.request("GET", "http://unresolvable.invalid/sdk")
57
+ self.assertEqual(client.getresponse().read(), b"through pinned proxy")
58
+ with socket.create_connection(("127.0.0.1", port), timeout=5) as client:
59
+ client.sendall(b"CONNECT unresolvable.invalid:443 HTTP/1.1\r\nHost: ignored\r\n\r\n")
60
+ self.assertIn(b"200", client.recv(4096))
61
+ client.sendall(b"test")
62
+ self.assertEqual(client.recv(4), b"test")
63
+ with self.assertRaises(OSError):
64
+ socket.create_connection(("127.0.0.1", port), timeout=0.2)
65
+ auth = "Basic " + base64.b64encode(b"fixture-user:fixture-pass").decode()
66
+ self.assertEqual(seen, [("GET", "http://unresolvable.invalid/sdk", auth),
67
+ ("CONNECT", "unresolvable.invalid:443", auth)])
68
+
69
+ def test_upstream_failure_returns_bounded_502_without_a_direct_request(self):
70
+ with socket.socket() as unused:
71
+ unused.bind(("127.0.0.1", 0))
72
+ closed_port = unused.getsockname()[1]
73
+ with relay.running(f"http://127.0.0.1:{closed_port}") as port:
74
+ with closing(http.client.HTTPConnection("127.0.0.1", port, timeout=5)) as client:
75
+ client.request("CONNECT", "unresolvable.invalid:443")
76
+ response = client.getresponse()
77
+ self.assertEqual((response.status, response.read()), (502, b""))
78
+
79
+ @unittest.skipUnless(shutil.which("java"), "Java runtime unavailable")
80
+ def test_actual_java_urlconnection_uses_explicit_proxy_with_no_credentials_in_options(self):
81
+ source = ('import java.net.*; public class Probe { public static void main(String[] args) '
82
+ 'throws Exception { var c = new URL("http://unresolvable.invalid/sdk").openConnection(); '
83
+ 'c.setConnectTimeout(3000); c.setReadTimeout(3000); '
84
+ 'System.out.print(new String(c.getInputStream().readAllBytes())); }}')
85
+ with tempfile.TemporaryDirectory() as directory, upstream() as (proxy, seen):
86
+ target = Path(directory) / "Probe.java"
87
+ target.write_text(source)
88
+ with relay.running(proxy) as port:
89
+ env = relay.environment({**os.environ, "JAVA_TOOL_OPTIONS": "-Dhttp.proxyHost=wrong.invalid"}, port)
90
+ result = subprocess.run(["java", str(target)], env=env, capture_output=True, text=True, timeout=20)
91
+ self.assertEqual(result.returncode, 0, result.stderr)
92
+ self.assertEqual(result.stdout, "through pinned proxy")
93
+ self.assertEqual(seen[0][1], "http://unresolvable.invalid/sdk")
94
+ self.assertNotIn("fixture-pass", result.stderr)
95
+ self.assertIn("-Dorg.gradle.daemon=false", env["GRADLE_OPTS"])
96
+
97
+
98
+ if __name__ == "__main__":
99
+ unittest.main()
@@ -57,6 +57,21 @@ class PlayProxyTests(unittest.TestCase):
57
57
  play_store_proxy.run(["true"])
58
58
  run.assert_not_called()
59
59
 
60
+ def test_failed_child_preserves_exit_code_and_closes_its_relay(self):
61
+ import json
62
+ import socket
63
+ import tempfile
64
+ with tempfile.TemporaryDirectory() as directory:
65
+ path = Path(directory) / "jvm-options.json"
66
+ probe = ("import json, os, sys; "
67
+ "open(sys.argv[1], 'w').write(json.dumps(os.environ['JAVA_TOOL_OPTIONS'])); sys.exit(7)")
68
+ with mock.patch.dict(os.environ, {"GOOGLE_STORE_PROXY_URL": "http://127.0.0.1:1"}):
69
+ self.assertEqual(play_store_proxy.run([sys.executable, "-c", probe, str(path)]), 7)
70
+ options = json.loads(path.read_text())
71
+ port = int(options.split("-Dhttp.proxyPort=", 1)[1].split()[0])
72
+ with self.assertRaises(OSError):
73
+ socket.create_connection(("127.0.0.1", port), timeout=0.2)
74
+
60
75
  def test_github_env_mode_clears_only_the_lowercase_bypass_for_later_steps(self):
61
76
  import tempfile
62
77
  value = "http://assigned.proxy.test:1234"
@@ -1 +1 @@
1
- 1.0.60
1
+ 1.0.62
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "gowalk-cicd",
3
- "version": "1.0.60",
3
+ "version": "1.0.62",
4
4
  "description": "Zero-config GitHub Actions delivery for iOS TestFlight and Android Google Play.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -286,7 +286,8 @@ jobs:
286
286
  echo "Before:"; df -h / | tail -1
287
287
  # NOT /usr/local/lib/node_modules: npm and npx live there, and the
288
288
  # Android action's Crashlytics symbol upload shells out to npx.
289
- sudo rm -rf /usr/share/dotnet /usr/local/lib/android/sdk/ndk \
289
+ # Keep installed Android SDK/NDKs: removing them makes AGP download them again.
290
+ sudo rm -rf /usr/share/dotnet \
290
291
  /opt/ghc /usr/local/.ghcup /usr/local/share/powershell \
291
292
  /usr/share/swift /usr/local/share/chromium 2>/dev/null || true
292
293
  sudo docker image prune --all --force >/dev/null 2>&1 || true