gowalk-cicd 1.0.58 → 1.0.60
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +21 -27
- package/README.md +21 -64
- package/action/.daemux-version +1 -1
- package/action/action.yml +25 -7
- package/action/scripts/crashlytics_dsyms.py +9 -31
- package/action/scripts/prepare_crashlytics_build.py +77 -0
- package/action/scripts/prepare_crashlytics_dsyms.py +59 -0
- package/action/scripts/test_crashlytics_dsyms.py +8 -11
- package/action/scripts/test_prepare_crashlytics.py +94 -0
- package/android-action/.daemux-version +1 -1
- package/android-action/action.yml +15 -2
- package/android-action/scripts/crashlytics_symbols.py +6 -3
- package/android-action/scripts/play_preflight.py +2 -1
- package/android-action/scripts/play_store_proxy.py +65 -2
- package/android-action/scripts/test_crashlytics_symbols.py +4 -1
- package/android-action/scripts/test_play_store_proxy.py +51 -0
- package/backend-action/.daemux-version +1 -1
- package/package.json +1 -1
- package/templates/deploy.yml +10 -4
package/CLAUDE.md
CHANGED
|
@@ -96,6 +96,11 @@ node /path/to/gowalk-cicd/bin/cli.mjs
|
|
|
96
96
|
- Keep the CLI surface minimal: `--dry-run`, `-v`, `-h`. Resist adding flags.
|
|
97
97
|
- `.gitignore` in the consumer repo is git-tracked. Never auto-edit it.
|
|
98
98
|
Detect and warn, tell the user to remove lines manually.
|
|
99
|
+
- Never repeat an `env:` key in another case (`NO_PROXY` beside `no_proxy`): GitHub
|
|
100
|
+
compares env keys case-insensitively and rejects the whole workflow or action file.
|
|
101
|
+
`test/env-keys.test.mjs` scans the templates and actions. Set lower-case spellings
|
|
102
|
+
or clear a lower-case bypass through `play_store_proxy.py -- <command>` (child
|
|
103
|
+
process) or `play_store_proxy.py --github-env` (later steps), never through YAML.
|
|
99
104
|
- Store API calls belong in GitHub Actions. Local tests may build/sign but must
|
|
100
105
|
never connect to App Store Connect or Google Play.
|
|
101
106
|
- Every Apple/Google store API call, OAuth exchange and binary upload requires its
|
|
@@ -161,31 +166,19 @@ node /path/to/gowalk-cicd/bin/cli.mjs
|
|
|
161
166
|
must not discard the symbols of a build that is already live. Symbol
|
|
162
167
|
artifacts carry no `retention-days` so the repository's setting governs and
|
|
163
168
|
can be raised to cover a release's life.
|
|
164
|
-
- Crashlytics
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
`action/scripts/crashlytics_dsyms.py` — because Flutter's `.symbols` are
|
|
178
|
-
ELF debug info the Firebase CLI's generators reject for Apple targets, and
|
|
179
|
-
`App.framework.dSYM` is the documented Apple path. Both run before the
|
|
180
|
-
store upload and fail closed — except an iOS id on an app with no
|
|
181
|
-
FirebaseCrashlytics `upload-symbols` tool, which is a warn-and-skip: an app
|
|
182
|
-
that does not embed the SDK has nothing to symbolicate. Logic belongs in
|
|
183
|
-
those scripts, where it is unit-tested, not inline in YAML. Supporting
|
|
184
|
-
invariants: the workflow's disk-space step must never remove
|
|
185
|
-
`/usr/local/lib/node_modules` (npm and npx live there); `firebase-tools` is
|
|
186
|
-
pinned exactly because the Android job holds the keystore and the Play
|
|
187
|
-
service account; the CLI runs from a temp cwd so its `.crashlytics/` debris
|
|
188
|
-
never lands in the checkout.
|
|
169
|
+
- Firebase Crashlytics traffic follows the Google account proxy. Android builds and
|
|
170
|
+
the pinned Firebase CLI receive the explicit proxy environment; Crashlytics Gradle
|
|
171
|
+
2.9.2+ and the pinned buildtools support it (generic JVM networking does not).
|
|
172
|
+
Native iOS upload-symbols uses NSURLSession, so it is never executed. Before
|
|
173
|
+
archiving, dedicated Crashlytics upload phases in the ephemeral Xcode projects
|
|
174
|
+
are deferred; mixed/unidentified upload phases refuse the archive for repair.
|
|
175
|
+
CI retains the archive's dSYMs as a ZIP with UUIDs, source SHA and SHA-256, then
|
|
176
|
+
emits `firebase_symbols_pending` (`gowalk-cicd/firebase-symbols-pending.v1`).
|
|
177
|
+
The app session downloads that exact artifact and uploads it through its
|
|
178
|
+
account-pinned Firebase console, reading processing back before completing the
|
|
179
|
+
delivery request. This requires no human handoff. A green CI run alone is not
|
|
180
|
+
evidence that the deferred symbols reached Firebase. Android retains its Dart
|
|
181
|
+
symbols and uploads them before the store bundle, using the configured proxy.
|
|
189
182
|
- The iOS action reads every script and prompt from the snapshot it takes of
|
|
190
183
|
itself in its first step (`$SWIFT_APP_ACTION`, under `RUNNER_TEMP`), never
|
|
191
184
|
from `${{ github.action_path }}` after that step. The plugin self-update
|
|
@@ -194,14 +187,15 @@ node /path/to/gowalk-cicd/bin/cli.mjs
|
|
|
194
187
|
release that fixes it; the snapshot is what keeps that run on one version.
|
|
195
188
|
`test_action_step_order.py` pins both. A new script reference in
|
|
196
189
|
`action/action.yml` therefore uses `"$SWIFT_APP_ACTION/scripts/..."`.
|
|
197
|
-
-
|
|
190
|
+
- Three situations end a green job with work for the panel, and each is a typed
|
|
198
191
|
check-run annotation (`::notice title=<name>::<one-line JSON with a versioned
|
|
199
192
|
schema>`), never prose in a `::warning::`: `play_review_pending`
|
|
200
193
|
(`templates/deploy.yml`, Play refused auto-submit and the bundle landed
|
|
201
194
|
unsubmitted) and `store_version_locked` (`action/action.yml`, the
|
|
202
195
|
`TESTFLIGHT_ONLY` decision of `manage_marketing_version.py`: an App Store
|
|
203
196
|
version is under review, so the build shipped to TestFlight under that
|
|
204
|
-
version and metadata was skipped)
|
|
197
|
+
version and metadata was skipped), and `firebase_symbols_pending` (the dSYM
|
|
198
|
+
archive awaits the session's proxied console upload). The README lists the schemas; add a
|
|
205
199
|
field by bumping the version.
|
|
206
200
|
- The JDK is chosen by `scripts/select_jdk.py`, called from the workflow before
|
|
207
201
|
`setup-java`. The `JAVA_VERSION` repo variable always wins; otherwise Flutter
|
package/README.md
CHANGED
|
@@ -279,6 +279,7 @@ the situation and whose **message** is one line of JSON carrying a versioned
|
|
|
279
279
|
| --- | --- | --- |
|
|
280
280
|
| `play_review_pending` | `gowalk-cicd/play-review-pending.v1` | `package`, `track`, `version_code` |
|
|
281
281
|
| `store_version_locked` | `gowalk-cicd/store-version-locked.v1` | `version`, `state`, `build_number` |
|
|
282
|
+
| `firebase_symbols_pending` | `gowalk-cicd/firebase-symbols-pending.v1` | `platform`, `app_id`, `source_sha`, `artifact`, `file`, `sha256`, `status` |
|
|
282
283
|
|
|
283
284
|
Add a field by bumping the schema version; never change the meaning of an
|
|
284
285
|
existing one.
|
|
@@ -329,7 +330,7 @@ The Android job removes the preinstalled toolchains a Flutter build never uses
|
|
|
329
330
|
(.NET, the Android NDK, GHC, PowerShell, Swift, Chromium) and prunes Docker
|
|
330
331
|
images, reclaiming roughly 25 GB in a few seconds. It prints `df -h /` before
|
|
331
332
|
and after. Linux only; skipped in Bitrise mode. npm survives on purpose: the
|
|
332
|
-
[Crashlytics symbol upload](#crashlytics-symbol-
|
|
333
|
+
[Crashlytics symbol upload](#crashlytics-symbol-delivery-firebase_app_id) runs
|
|
333
334
|
the Firebase CLI through `npx`.
|
|
334
335
|
|
|
335
336
|
### Private git dependencies (Flutter)
|
|
@@ -572,72 +573,28 @@ the workflow uploads them itself once the repository variable
|
|
|
572
573
|
**`FIREBASE_APP_ID`** is set — see below. Sentry or others: upload the same
|
|
573
574
|
directory to that service from your own pipeline.
|
|
574
575
|
|
|
575
|
-
#### Crashlytics symbol
|
|
576
|
+
#### Crashlytics symbol delivery (`FIREBASE_APP_ID`)
|
|
576
577
|
|
|
577
|
-
Set the repository variable
|
|
578
|
-
`1:<project
|
|
579
|
-
|
|
578
|
+
Set the repository variable to the Firebase App IDs for the shipping platforms
|
|
579
|
+
(`1:<project>:ios:<hash>` and/or `1:<project>:android:<hash>`). The workflow passes it
|
|
580
|
+
as the `firebase-app-id` input. Configure the account's `GOOGLE_STORE_PROXY_URL`
|
|
581
|
+
repository secret before any provider operation.
|
|
580
582
|
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
583
|
+
Android builds and the pinned Firebase CLI receive the proxy explicitly, including
|
|
584
|
+
Crashlytics mapping/native-symbol uploads. Crashlytics Gradle 2.9.2+ supports proxy
|
|
585
|
+
environment variables; older versions must be updated before use. Dart symbols are
|
|
586
|
+
retained as artifacts and uploaded through that transport before the bundle ships.
|
|
584
587
|
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
`ios
|
|
588
|
-
|
|
589
|
-
|
|
588
|
+
Native iOS `upload-symbols` is refused because it does not use this environment
|
|
589
|
+
transport. CI defers dedicated native upload phases in its ephemeral projects and
|
|
590
|
+
retains `ios-crashlytics-symbols-<run>-<attempt>` with `ios-dsyms.zip` and a manifest
|
|
591
|
+
containing UUIDs, source SHA and SHA-256. The typed `firebase_symbols_pending`
|
|
592
|
+
annotation uses schema `gowalk-cicd/firebase-symbols-pending.v1` and names the exact
|
|
593
|
+
artifact. The autonomous app session must upload it through the account-pinned
|
|
594
|
+
Firebase console's Crashlytics dSYMs tab and read back processing; no human action
|
|
595
|
+
is required. A completed CI run does not mean this deferred upload is complete.
|
|
596
|
+
[Firebase documents the ZIP upload interface](https://firebase.google.com/docs/crashlytics/ios/get-deobfuscated-reports).
|
|
590
597
|
|
|
591
|
-
The variable reaches the actions through `deploy.yml`, which
|
|
592
|
-
[auto-update never touches](#deployyml-is-not-auto-updated) — an existing repo
|
|
593
|
-
must run `npx --yes gowalk-cicd` once before setting it does anything.
|
|
594
|
-
|
|
595
|
-
| | Android | iOS |
|
|
596
|
-
|---|---|---|
|
|
597
|
-
| What is uploaded | the `--split-debug-info` directory, with the command Firebase documents for Flutter: `firebase crashlytics:symbols:upload --app=<android id> <dir>` (Firebase CLI through `npx`, major-pinned) | every dSYM in the archive — `App.framework.dSYM` carries the Dart frames — with the `upload-symbols` tool from the FirebaseCrashlytics pod: `upload-symbols -ai <ios id> -p ios -- <archive>/dSYMs` |
|
|
598
|
-
| When | in the Android action, right after the build and before the bundle goes to Play | in the iOS action, right after the IPA export and before the TestFlight upload |
|
|
599
|
-
| If it fails | the deploy fails; nothing ships with crashes nobody can read. The symbols are still in the `android-symbols-*` artifact for a manual upload | same — the failure lands before the TestFlight upload, so nothing shipped; fix the cause and re-run the deploy |
|
|
600
|
-
| Variable unset | a `::warning::` when the app depends on `firebase_crashlytics` (every Dart frame will be `***`) | nothing — the run-script phase that `flutterfire configure` adds already uploads `App.framework.dSYM`, and the workflow installs the flutterfire CLI it needs |
|
|
601
|
-
| Variable set for the other platform only | a `::warning::`, no upload | a `::notice::`, no upload |
|
|
602
|
-
|
|
603
|
-
Why the two platforms differ: Flutter's `.symbols` files are ELF debug info,
|
|
604
|
-
which the Firebase CLI's symbol generators accept for Android and reject for
|
|
605
|
-
Apple targets; Firebase documents dSYMs as the Apple path, and Flutter 3.12+
|
|
606
|
-
puts the Dart debug info into `App.framework.dSYM`. On iOS the explicit
|
|
607
|
-
upload is a belt-and-braces over the run-script phase — it also covers the
|
|
608
|
-
native `Runner.app.dSYM` and every plugin framework — so a phase that is
|
|
609
|
-
missing or failing can never leave a build unreadable.
|
|
610
|
-
|
|
611
|
-
A value that is not a Firebase App ID fails the deploy rather than being
|
|
612
|
-
skipped: a typo must not become a fleet that silently stops uploading symbols.
|
|
613
|
-
An iOS id on an app that does not embed Crashlytics (no `upload-symbols` tool
|
|
614
|
-
anywhere under the project) is a warning, not a failure — an app without the
|
|
615
|
-
SDK has no Crashlytics crashes to symbolicate. Android apps delivered through
|
|
616
|
-
Bitrise are built by Bitrise and are not covered.
|
|
617
|
-
|
|
618
|
-
Obfuscation renames identifiers, so code that depends on their spelling breaks
|
|
619
|
-
at runtime, not at build time. Do not rely on `runtimeType.toString()`,
|
|
620
|
-
`Type.toString()`, or on matching class or function names in stack traces
|
|
621
|
-
(`Foo` becomes `Lk`, and a raw trace is a row of `***`) — compare types with
|
|
622
|
-
`is` instead. Enum values keep their names today (`Color.red.toString()` is
|
|
623
|
-
still `Color.red`), but `flutter build --help` lists `Enum.toString()` among
|
|
624
|
-
the methods that may return obfuscated results, so prefer `Enum.name` when the
|
|
625
|
-
identifier matters.
|
|
626
|
-
|
|
627
|
-
Where the flags are applied: Android passes them straight to
|
|
628
|
-
`flutter build appbundle`. iOS is less obvious — `deploy.yml` runs
|
|
629
|
-
`flutter build ios --config-only` with the flags, which writes
|
|
630
|
-
`DART_OBFUSCATION=true` and `SPLIT_DEBUG_INFO=<dir>` into
|
|
631
|
-
`ios/Flutter/Generated.xcconfig`, and the Flutter build phase inside the
|
|
632
|
-
`xcodebuild archive` that follows compiles against those settings. The symbol
|
|
633
|
-
directory lives under the runner's temp directory on both platforms, never in
|
|
634
|
-
the checkout.
|
|
635
|
-
|
|
636
|
-
Because the iOS half lives in `deploy.yml`, which
|
|
637
|
-
[auto-update never touches](#deployyml-is-not-auto-updated), an existing
|
|
638
|
-
Flutter repo gets Android obfuscation on its next auto-update but iOS
|
|
639
|
-
obfuscation only after a one-off `npx --yes gowalk-cicd`. The Android action
|
|
640
|
-
retains its own symbols precisely so that this half-updated state is safe.
|
|
641
598
|
|
|
642
599
|
### iOS delivery
|
|
643
600
|
|
|
@@ -975,7 +932,7 @@ Changes that need that manual run:
|
|
|
975
932
|
`ios-symbols-*`. See [Dart obfuscation](#dart-obfuscation-flutter-apps).
|
|
976
933
|
- **Crashlytics symbol upload** — `firebase-app-id: ${{ vars.FIREBASE_APP_ID }}`
|
|
977
934
|
passed to both actions. Until it is, setting the variable does nothing.
|
|
978
|
-
See [Crashlytics symbol upload](#crashlytics-symbol-
|
|
935
|
+
See [Crashlytics symbol upload](#crashlytics-symbol-delivery-firebase_app_id).
|
|
979
936
|
|
|
980
937
|
### Opt out
|
|
981
938
|
|
package/action/.daemux-version
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.60
|
package/action/action.yml
CHANGED
|
@@ -693,6 +693,16 @@ runs:
|
|
|
693
693
|
|| /usr/libexec/PlistBuddy -c "Add :ITSAppUsesNonExemptEncryption bool $VALUE" "$INFOPLIST_PATH"
|
|
694
694
|
echo "Set ITSAppUsesNonExemptEncryption=$VALUE in $INFOPLIST_PATH"
|
|
695
695
|
|
|
696
|
+
- name: Defer native Firebase upload phases to the proxied console
|
|
697
|
+
if: ${{ inputs.archive == 'true' }}
|
|
698
|
+
shell: bash
|
|
699
|
+
env:
|
|
700
|
+
PROJECT: ${{ inputs.project || env.CFG_PROJECT }}
|
|
701
|
+
WORKSPACE: ${{ inputs.workspace || env.CFG_WORKSPACE }}
|
|
702
|
+
run: |
|
|
703
|
+
python3 "$SWIFT_APP_ACTION/scripts/prepare_crashlytics_build.py" \
|
|
704
|
+
--search-root "$(dirname "${WORKSPACE:-$PROJECT}")"
|
|
705
|
+
|
|
696
706
|
- name: Archive
|
|
697
707
|
if: ${{ inputs.archive == 'true' }}
|
|
698
708
|
shell: bash
|
|
@@ -787,20 +797,28 @@ runs:
|
|
|
787
797
|
# set (deploy.yml passes the FIREBASE_APP_ID repository variable) every
|
|
788
798
|
# dSYM in the archive is uploaded here as well, with the upload-symbols
|
|
789
799
|
# tool from the FirebaseCrashlytics pod, so a missing or broken phase
|
|
790
|
-
# cannot
|
|
791
|
-
#
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
if: ${{ inputs.archive == 'true' && inputs.upload == 'true' && inputs.firebase-app-id != '' }}
|
|
800
|
+
# cannot lose the exact build's symbols. The autonomous session completes
|
|
801
|
+
# the upload through the proxied Firebase console from this typed artifact.
|
|
802
|
+
- name: Retain dSYMs for the proxied Firebase console
|
|
803
|
+
if: ${{ inputs.archive == 'true' && inputs.upload == 'true' }}
|
|
795
804
|
shell: bash
|
|
796
805
|
env:
|
|
797
806
|
FIREBASE_APP_ID: ${{ inputs.firebase-app-id }}
|
|
798
807
|
PROJECT: ${{ inputs.project || env.CFG_PROJECT }}
|
|
799
808
|
WORKSPACE: ${{ inputs.workspace || env.CFG_WORKSPACE }}
|
|
800
809
|
run: |
|
|
801
|
-
python3 "$SWIFT_APP_ACTION/scripts/
|
|
810
|
+
python3 "$SWIFT_APP_ACTION/scripts/prepare_crashlytics_dsyms.py" \
|
|
802
811
|
--archive "$RUNNER_TEMP/app.xcarchive" \
|
|
803
|
-
--search-root "$(dirname "${WORKSPACE:-$PROJECT}")"
|
|
812
|
+
--search-root "$(dirname "${WORKSPACE:-$PROJECT}")" \
|
|
813
|
+
--output "$RUNNER_TEMP/firebase-symbols"
|
|
814
|
+
|
|
815
|
+
- name: Preserve the Firebase console upload artifact
|
|
816
|
+
if: ${{ always() && inputs.archive == 'true' }}
|
|
817
|
+
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4
|
|
818
|
+
with:
|
|
819
|
+
name: ios-crashlytics-symbols-${{ github.run_id }}-${{ github.run_attempt }}
|
|
820
|
+
path: ${{ runner.temp }}/firebase-symbols/
|
|
821
|
+
if-no-files-found: ignore
|
|
804
822
|
|
|
805
823
|
- name: Upload to TestFlight
|
|
806
824
|
if: ${{ inputs.archive == 'true' && inputs.upload == 'true' }}
|
|
@@ -1,28 +1,9 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
|
-
"""
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
--split-debug-info builds (the .symbols files those flags write are ELF
|
|
8
|
-
debug info that the Firebase CLI's symbol generators reject). The run-script
|
|
9
|
-
phase `flutterfire configure` adds already uploads that one dSYM during the
|
|
10
|
-
archive; this script uploads all of them (App.framework, the app binary,
|
|
11
|
-
plugin frameworks) with the `upload-symbols` tool that ships in the
|
|
12
|
-
FirebaseCrashlytics pod, so a missing or broken phase cannot leave a build
|
|
13
|
-
unreadable, and native frames get their symbols too.
|
|
14
|
-
|
|
15
|
-
upload-symbols -ai <FIREBASE_APP_ID> -p ios -- <archive>/dSYMs
|
|
16
|
-
|
|
17
|
-
No credentials: the tool authenticates by app id alone.
|
|
18
|
-
|
|
19
|
-
Gating: FIREBASE_APP_ID holds the app's Firebase App ID(s),
|
|
20
|
-
`1:<project number>:<platform>:<hash>`, comma- or whitespace-separated when an
|
|
21
|
-
app ships both platforms; the iOS one is used and its absence is a notice, not
|
|
22
|
-
a failure (the run-script phase is the documented default). With an iOS id
|
|
23
|
-
present, a missing tool or a failed upload fails the step — the action runs it
|
|
24
|
-
before the TestFlight upload, so a deploy never goes out with crashes nobody
|
|
25
|
-
can read.
|
|
2
|
+
"""Shared Crashlytics app/tool discovery and the retired native uploader interface.
|
|
3
|
+
|
|
4
|
+
CI uses prepare_crashlytics_dsyms.py to retain an archive and a typed console-upload
|
|
5
|
+
receipt. Native upload-symbols uses NSURLSession without the task proxy environment,
|
|
6
|
+
so executing it from this legacy interface is refused. No direct fallback is allowed.
|
|
26
7
|
"""
|
|
27
8
|
|
|
28
9
|
from __future__ import annotations
|
|
@@ -104,11 +85,8 @@ def upload_command(tool: Path, app_id: str, dsyms_dir: Path) -> list[str]:
|
|
|
104
85
|
|
|
105
86
|
|
|
106
87
|
def _default_run(cmd: list[str]) -> int:
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
except OSError as exc:
|
|
110
|
-
print(f"::error::could not run {cmd[0]}: {exc}")
|
|
111
|
-
return 126
|
|
88
|
+
raise SystemExit("Native Crashlytics upload is refused; use prepare_crashlytics_dsyms.py "
|
|
89
|
+
"and complete the upload through the account-proxied console")
|
|
112
90
|
|
|
113
91
|
|
|
114
92
|
def main(argv: list[str] | None = None, environ: dict[str, str] | None = None, run=None,
|
|
@@ -134,10 +112,10 @@ def main(argv: list[str] | None = None, environ: dict[str, str] | None = None, r
|
|
|
134
112
|
if ids:
|
|
135
113
|
print(
|
|
136
114
|
"::notice::FIREBASE_APP_ID names no iOS app (1:<project number>:ios:<hash>); "
|
|
137
|
-
"
|
|
115
|
+
"native upload is disabled; use the proxied console delivery."
|
|
138
116
|
)
|
|
139
117
|
else:
|
|
140
|
-
print("FIREBASE_APP_ID unset;
|
|
118
|
+
print("FIREBASE_APP_ID unset; native upload is disabled; configure proxied console delivery.")
|
|
141
119
|
return 0
|
|
142
120
|
|
|
143
121
|
dsyms_dir = args.archive / "dSYMs"
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Defer dedicated native Crashlytics upload phases to the proxied console delivery."""
|
|
3
|
+
from __future__ import annotations
|
|
4
|
+
|
|
5
|
+
import argparse
|
|
6
|
+
import json
|
|
7
|
+
from pathlib import Path
|
|
8
|
+
import plistlib
|
|
9
|
+
import re
|
|
10
|
+
import shlex
|
|
11
|
+
import subprocess
|
|
12
|
+
|
|
13
|
+
MARKER = "echo 'Crashlytics dSYMs are retained by CI for the account-proxied console upload'"
|
|
14
|
+
UPLOADS = ("upload-crashlytics-symbols", "upload-symbols", "Crashlytics/run")
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
def dedicated(script: str) -> bool:
|
|
18
|
+
lines = [line.strip() for line in script.replace("\\\n", " ").splitlines()
|
|
19
|
+
if line.strip() and not line.lstrip().startswith("#")]
|
|
20
|
+
commands = [line for line in lines if not re.fullmatch(r'(?:export )?PATH=[^;&|<>`]+', line)]
|
|
21
|
+
if len(commands) != 1 or any(token in script for token in ("$(", "`")):
|
|
22
|
+
return False
|
|
23
|
+
lexer = shlex.shlex(commands[0], posix=True, punctuation_chars=";&|<>")
|
|
24
|
+
lexer.whitespace_split = True
|
|
25
|
+
tokens = list(lexer)
|
|
26
|
+
if not tokens or any(set(token) <= set(";&|<>") for token in tokens):
|
|
27
|
+
return False
|
|
28
|
+
return (tokens[0].endswith(("upload-symbols", "Crashlytics/run"))
|
|
29
|
+
or (tokens[0] == "flutterfire" and tokens[1:2] == ["upload-crashlytics-symbols"]))
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
def rewrite(document: dict) -> list[str]:
|
|
33
|
+
changed = []
|
|
34
|
+
for identity, phase in document.get("objects", {}).items():
|
|
35
|
+
if phase.get("isa") != "PBXShellScriptBuildPhase":
|
|
36
|
+
continue
|
|
37
|
+
script = phase.get("shellScript", "")
|
|
38
|
+
active = "\n".join(line for line in script.splitlines() if not line.lstrip().startswith("#"))
|
|
39
|
+
if not any(token in active for token in UPLOADS):
|
|
40
|
+
continue
|
|
41
|
+
if "crashlytics" not in str(phase.get("name", "")).lower() or not dedicated(script):
|
|
42
|
+
raise ValueError("Crashlytics upload is in a mixed build phase; separate it before archiving")
|
|
43
|
+
phase["shellScript"] = MARKER
|
|
44
|
+
changed.append(identity)
|
|
45
|
+
return changed
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
def prepare(root: Path) -> int:
|
|
49
|
+
root = root.resolve()
|
|
50
|
+
projects = sorted(root.rglob("project.pbxproj"))
|
|
51
|
+
prepared = []
|
|
52
|
+
for project in projects:
|
|
53
|
+
if project.is_symlink() or root not in project.resolve().parents:
|
|
54
|
+
raise ValueError("Xcode project leaves the CI workspace")
|
|
55
|
+
raw = subprocess.check_output(["plutil", "-convert", "json", "-o", "-", str(project)], timeout=30)
|
|
56
|
+
document = json.loads(raw)
|
|
57
|
+
changed = rewrite(document)
|
|
58
|
+
if changed:
|
|
59
|
+
prepared.append((project, document, changed))
|
|
60
|
+
# Validate every project before changing one; a mixed phase cannot produce
|
|
61
|
+
# a partially rewritten workspace that a caller accidentally archives.
|
|
62
|
+
for project, document, changed in prepared:
|
|
63
|
+
temporary = project.with_suffix(".proxy-new")
|
|
64
|
+
try:
|
|
65
|
+
temporary.write_bytes(plistlib.dumps(document, sort_keys=False))
|
|
66
|
+
temporary.replace(project)
|
|
67
|
+
finally:
|
|
68
|
+
temporary.unlink(missing_ok=True)
|
|
69
|
+
print(f"Deferred {len(changed)} Crashlytics upload phase(s) in {project.relative_to(root)}")
|
|
70
|
+
return sum(len(changed) for _, _, changed in prepared)
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
if __name__ == "__main__":
|
|
74
|
+
parser = argparse.ArgumentParser(description=__doc__)
|
|
75
|
+
parser.add_argument("--search-root", required=True, type=Path)
|
|
76
|
+
args = parser.parse_args()
|
|
77
|
+
prepare(args.search_root)
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Retain iOS dSYMs and a typed receipt for automatic upload through the proxied console."""
|
|
3
|
+
from __future__ import annotations
|
|
4
|
+
|
|
5
|
+
import argparse
|
|
6
|
+
import hashlib
|
|
7
|
+
import json
|
|
8
|
+
import os
|
|
9
|
+
from pathlib import Path
|
|
10
|
+
import subprocess
|
|
11
|
+
import zipfile
|
|
12
|
+
|
|
13
|
+
from crashlytics_dsyms import find_upload_symbols, select_app_id
|
|
14
|
+
|
|
15
|
+
SCHEMA = "gowalk-cicd/firebase-symbols-pending.v1"
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
def prepare(archive: Path, output: Path, app_id: str) -> dict:
|
|
19
|
+
root = (archive / "dSYMs").resolve()
|
|
20
|
+
bundles = sorted(root.glob("*.dSYM"))
|
|
21
|
+
files = sorted(path for bundle in bundles for path in bundle.rglob("*") if path.is_file())
|
|
22
|
+
if not files or any(path.is_symlink() or root not in path.resolve().parents for path in files):
|
|
23
|
+
raise ValueError("Archive must contain dSYM files owned by this build")
|
|
24
|
+
output.mkdir(parents=True, exist_ok=True)
|
|
25
|
+
target = output / "ios-dsyms.zip"
|
|
26
|
+
with zipfile.ZipFile(target, "w", compression=zipfile.ZIP_DEFLATED) as zipped:
|
|
27
|
+
for path in files:
|
|
28
|
+
zipped.write(path, path.relative_to(root))
|
|
29
|
+
with target.open("rb") as stream:
|
|
30
|
+
digest = hashlib.file_digest(stream, "sha256").hexdigest()
|
|
31
|
+
uuids = subprocess.check_output(["dwarfdump", "--uuid", *(str(bundle) for bundle in bundles)],
|
|
32
|
+
text=True, timeout=60).splitlines()
|
|
33
|
+
artifact = f"ios-crashlytics-symbols-{os.environ['GITHUB_RUN_ID']}-{os.environ['GITHUB_RUN_ATTEMPT']}"
|
|
34
|
+
receipt = {"schema": SCHEMA, "platform": "ios", "app_id": app_id,
|
|
35
|
+
"source_sha": os.environ["GITHUB_SHA"], "artifact": artifact,
|
|
36
|
+
"file": target.name, "sha256": digest, "status": "pending_console_upload"}
|
|
37
|
+
(output / "firebase-symbols.json").write_text(json.dumps({**receipt, "uuids": uuids}, indent=2))
|
|
38
|
+
return receipt
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
def main() -> None:
|
|
42
|
+
parser = argparse.ArgumentParser(description=__doc__)
|
|
43
|
+
parser.add_argument("--archive", required=True, type=Path)
|
|
44
|
+
parser.add_argument("--search-root", required=True, type=Path)
|
|
45
|
+
parser.add_argument("--output", required=True, type=Path)
|
|
46
|
+
args = parser.parse_args()
|
|
47
|
+
app_id = select_app_id(os.environ.get("FIREBASE_APP_ID", ""), "ios")
|
|
48
|
+
tool = find_upload_symbols(args.search_root)
|
|
49
|
+
if not tool:
|
|
50
|
+
print("No Crashlytics SDK upload tool found; no Firebase symbol delivery is required.")
|
|
51
|
+
return
|
|
52
|
+
if not app_id:
|
|
53
|
+
raise SystemExit("The app embeds Crashlytics but FIREBASE_APP_ID names no iOS app")
|
|
54
|
+
receipt = prepare(args.archive, args.output, app_id)
|
|
55
|
+
print("::notice title=firebase_symbols_pending::" + json.dumps(receipt, separators=(",", ":")))
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
if __name__ == "__main__":
|
|
59
|
+
main()
|
|
@@ -151,12 +151,9 @@ class MainTest(unittest.TestCase):
|
|
|
151
151
|
self.assertNotIn("::error::", out)
|
|
152
152
|
self.assertIn("upload-symbols", out)
|
|
153
153
|
|
|
154
|
-
def
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
rc = cd._default_run([str(self.root / "gowalk-cicd-no-such-binary")])
|
|
158
|
-
self.assertNotEqual(rc, 0)
|
|
159
|
-
self.assertIn("::error::", out.getvalue())
|
|
154
|
+
def test_native_tool_is_refused_without_executing_it(self) -> None:
|
|
155
|
+
with self.assertRaisesRegex(SystemExit, "account-proxied console"):
|
|
156
|
+
cd._default_run([str(self.root / "upload-symbols")])
|
|
160
157
|
|
|
161
158
|
def test_missing_dsyms_fail_before_any_upload(self) -> None:
|
|
162
159
|
code, out = self.run_main({"FIREBASE_APP_ID": IOS}, archive=self.root / "missing.xcarchive")
|
|
@@ -193,18 +190,18 @@ class CrashlyticsWiringTest(unittest.TestCase):
|
|
|
193
190
|
self.assertNotIn("vars.", text)
|
|
194
191
|
|
|
195
192
|
def test_dsyms_go_up_after_export_and_before_testflight(self) -> None:
|
|
196
|
-
upload = step("
|
|
193
|
+
upload = step("Retain dSYMs for the proxied Firebase console")
|
|
197
194
|
self.assertIn("inputs.archive == 'true'", upload)
|
|
198
195
|
# Only builds that actually ship: an archive-only run (upload=false)
|
|
199
196
|
# must not be failed over a symbols upload.
|
|
200
197
|
self.assertIn("inputs.upload == 'true'", upload)
|
|
201
|
-
self.
|
|
198
|
+
self.assertNotIn("inputs.firebase-app-id != ''", upload)
|
|
202
199
|
self.assertIn("FIREBASE_APP_ID: ${{ inputs.firebase-app-id }}", upload)
|
|
203
|
-
self.assertIn("scripts/
|
|
200
|
+
self.assertIn("scripts/prepare_crashlytics_dsyms.py", upload)
|
|
204
201
|
self.assertIn('--archive "$RUNNER_TEMP/app.xcarchive"', upload)
|
|
205
202
|
text = ACTION_YML.read_text()
|
|
206
|
-
self.assertLess(text.index("- name: Export IPA"), text.index("- name:
|
|
207
|
-
self.assertLess(text.index("- name:
|
|
203
|
+
self.assertLess(text.index("- name: Export IPA"), text.index("- name: Retain dSYMs for the proxied Firebase console"))
|
|
204
|
+
self.assertLess(text.index("- name: Retain dSYMs for the proxied Firebase console"), text.index("- name: Upload to TestFlight"))
|
|
208
205
|
|
|
209
206
|
def test_the_script_ships_with_the_action(self) -> None:
|
|
210
207
|
self.assertTrue((ACTION_YML.parent / "scripts" / "crashlytics_dsyms.py").is_file())
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
"""Native upload fencing and durable symbols preserve the exact archive without provider calls."""
|
|
2
|
+
import hashlib
|
|
3
|
+
import json
|
|
4
|
+
import os
|
|
5
|
+
from pathlib import Path
|
|
6
|
+
import sys
|
|
7
|
+
import tempfile
|
|
8
|
+
import unittest
|
|
9
|
+
from unittest import mock
|
|
10
|
+
import zipfile
|
|
11
|
+
|
|
12
|
+
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
|
13
|
+
import crashlytics_dsyms
|
|
14
|
+
import prepare_crashlytics_build as build
|
|
15
|
+
import prepare_crashlytics_dsyms as symbols
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
class BuildPhaseTests(unittest.TestCase):
|
|
19
|
+
def test_dedicated_phase_is_deferred_without_changing_build_objects(self):
|
|
20
|
+
other = {"isa": "PBXSourcesBuildPhase", "files": ["source"]}
|
|
21
|
+
doc = {"objects": {"code": other, "symbols": {"isa": "PBXShellScriptBuildPhase",
|
|
22
|
+
"name": "Crashlytics Upload Symbols", "shellScript": '"$PODS_ROOT/FirebaseCrashlytics/run"'}}}
|
|
23
|
+
self.assertEqual(build.rewrite(doc), ["symbols"])
|
|
24
|
+
self.assertEqual(doc["objects"]["code"], other)
|
|
25
|
+
self.assertEqual(doc["objects"]["symbols"]["shellScript"], build.MARKER)
|
|
26
|
+
self.assertEqual(build.rewrite(doc), [])
|
|
27
|
+
|
|
28
|
+
def test_unidentified_upload_phase_refuses_the_archive(self):
|
|
29
|
+
doc = {"objects": {"mixed": {"isa": "PBXShellScriptBuildPhase", "name": "Build everything",
|
|
30
|
+
"shellScript": "compile-app; upload-symbols"}}}
|
|
31
|
+
with self.assertRaises(ValueError):
|
|
32
|
+
build.rewrite(doc)
|
|
33
|
+
|
|
34
|
+
def test_crashlytics_name_cannot_hide_other_build_work(self):
|
|
35
|
+
doc = {"objects": {"mixed": {"isa": "PBXShellScriptBuildPhase", "name": "Crashlytics",
|
|
36
|
+
"shellScript": "compile-app; upload-symbols"}}}
|
|
37
|
+
with self.assertRaises(ValueError):
|
|
38
|
+
build.rewrite(doc)
|
|
39
|
+
self.assertTrue(build.dedicated('#!/bin/sh\nPATH="$PATH:$HOME/bin"\n'
|
|
40
|
+
'flutterfire upload-crashlytics-symbols --platform=ios'))
|
|
41
|
+
|
|
42
|
+
def test_legacy_native_entrypoint_never_starts_the_binary(self):
|
|
43
|
+
with mock.patch.object(crashlytics_dsyms.subprocess, "run") as run:
|
|
44
|
+
with self.assertRaises(SystemExit):
|
|
45
|
+
crashlytics_dsyms._default_run(["upload-symbols"])
|
|
46
|
+
run.assert_not_called()
|
|
47
|
+
|
|
48
|
+
def test_every_project_is_validated_before_any_file_changes(self):
|
|
49
|
+
with tempfile.TemporaryDirectory() as temporary:
|
|
50
|
+
root = Path(temporary)
|
|
51
|
+
first, second = root / "A.xcodeproj", root / "B.xcodeproj"
|
|
52
|
+
first.mkdir(); second.mkdir()
|
|
53
|
+
for parent in (first, second):
|
|
54
|
+
(parent / "project.pbxproj").write_text("original")
|
|
55
|
+
documents = [json.dumps({"objects": {"phase": {"isa": "PBXShellScriptBuildPhase",
|
|
56
|
+
"name": name, "shellScript": "upload-symbols"}}}).encode()
|
|
57
|
+
for name in ("Crashlytics", "Mixed")]
|
|
58
|
+
with mock.patch.object(build.subprocess, "check_output", side_effect=documents):
|
|
59
|
+
with self.assertRaises(ValueError):
|
|
60
|
+
build.prepare(root)
|
|
61
|
+
self.assertEqual((first / "project.pbxproj").read_text(), "original")
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
@mock.patch.dict(os.environ, {"GITHUB_RUN_ID": "123", "GITHUB_RUN_ATTEMPT": "2", "GITHUB_SHA": "source-sha"})
|
|
65
|
+
class ArchiveTests(unittest.TestCase):
|
|
66
|
+
def test_zip_manifest_and_notice_identify_the_same_build(self):
|
|
67
|
+
with tempfile.TemporaryDirectory() as temporary:
|
|
68
|
+
root = Path(temporary)
|
|
69
|
+
dwarf = root / "app.xcarchive/dSYMs/App.framework.dSYM/Contents/Resources/DWARF/App"
|
|
70
|
+
dwarf.parent.mkdir(parents=True); dwarf.write_bytes(b"actual debug symbols")
|
|
71
|
+
output = root / "output"
|
|
72
|
+
with mock.patch.object(symbols.subprocess, "check_output", return_value="UUID: expected (arm64) App\n"):
|
|
73
|
+
receipt = symbols.prepare(root / "app.xcarchive", output, "1:2:ios:abc")
|
|
74
|
+
manifest = json.loads((output / "firebase-symbols.json").read_text())
|
|
75
|
+
self.assertEqual(receipt["source_sha"], "source-sha")
|
|
76
|
+
self.assertEqual(receipt["artifact"], "ios-crashlytics-symbols-123-2")
|
|
77
|
+
self.assertEqual(receipt["sha256"], hashlib.sha256((output / "ios-dsyms.zip").read_bytes()).hexdigest())
|
|
78
|
+
self.assertEqual(manifest["uuids"], ["UUID: expected (arm64) App"])
|
|
79
|
+
with zipfile.ZipFile(output / "ios-dsyms.zip") as zipped:
|
|
80
|
+
self.assertEqual(zipped.read(zipped.namelist()[0]), b"actual debug symbols")
|
|
81
|
+
|
|
82
|
+
def test_symlink_to_another_owner_cannot_enter_the_artifact(self):
|
|
83
|
+
with tempfile.TemporaryDirectory() as temporary:
|
|
84
|
+
root = Path(temporary)
|
|
85
|
+
other = root / "other"; other.write_bytes(b"private")
|
|
86
|
+
dsym = root / "app.xcarchive/dSYMs/App.dSYM"; dsym.mkdir(parents=True)
|
|
87
|
+
(dsym / "outside").symlink_to(other)
|
|
88
|
+
with self.assertRaises(ValueError):
|
|
89
|
+
symbols.prepare(root / "app.xcarchive", root / "output", "1:2:ios:abc")
|
|
90
|
+
self.assertFalse((root / "output").exists())
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
if __name__ == "__main__":
|
|
94
|
+
unittest.main()
|
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.60
|
|
@@ -128,6 +128,12 @@ runs:
|
|
|
128
128
|
if: ${{ steps.config.outputs.project_kind == 'flutter' }}
|
|
129
129
|
shell: bash
|
|
130
130
|
env:
|
|
131
|
+
# Only one spelling per key: GitHub compares env keys case-insensitively and
|
|
132
|
+
# rejects the file otherwise. play_store_proxy.py -- runs the build with both
|
|
133
|
+
# spellings set and both NO_PROXY spellings cleared for the child process.
|
|
134
|
+
HTTPS_PROXY: ${{ env.GOOGLE_STORE_PROXY_URL }}
|
|
135
|
+
HTTP_PROXY: ${{ env.GOOGLE_STORE_PROXY_URL }}
|
|
136
|
+
NO_PROXY: ''
|
|
131
137
|
BUILD_NAME: ${{ inputs.build-name }}
|
|
132
138
|
DART_DEFINES: ${{ inputs.dart-defines }}
|
|
133
139
|
DART_SYMBOLS_DIR: ${{ runner.temp }}/dart-symbols/android
|
|
@@ -151,7 +157,7 @@ runs:
|
|
|
151
157
|
*) echo "::error::dart-defines entry '$define' is not KEY=VALUE"; exit 1 ;;
|
|
152
158
|
esac
|
|
153
159
|
done
|
|
154
|
-
flutter "${args[@]}"
|
|
160
|
+
python3 "${{ github.action_path }}/scripts/play_store_proxy.py" -- flutter "${args[@]}"
|
|
155
161
|
if ! ls "$DART_SYMBOLS_DIR"/*.symbols >/dev/null 2>&1; then
|
|
156
162
|
echo "::error::flutter build wrote no Dart symbol files to $DART_SYMBOLS_DIR;" \
|
|
157
163
|
"a crash from this obfuscated build could never be read. Refusing to ship it."
|
|
@@ -227,10 +233,17 @@ runs:
|
|
|
227
233
|
shell: bash
|
|
228
234
|
working-directory: ${{ steps.config.outputs.gradle_root || '.' }}
|
|
229
235
|
env:
|
|
236
|
+
# Only one spelling per key: GitHub compares env keys case-insensitively and
|
|
237
|
+
# rejects the file otherwise. play_store_proxy.py -- runs the build with both
|
|
238
|
+
# spellings set and both NO_PROXY spellings cleared for the child process.
|
|
239
|
+
HTTPS_PROXY: ${{ env.GOOGLE_STORE_PROXY_URL }}
|
|
240
|
+
HTTP_PROXY: ${{ env.GOOGLE_STORE_PROXY_URL }}
|
|
241
|
+
NO_PROXY: ''
|
|
230
242
|
ANDROID_BUILD_NAME: ${{ inputs.build-name }}
|
|
231
243
|
run: |
|
|
232
244
|
chmod +x ./gradlew
|
|
233
|
-
|
|
245
|
+
python3 "${{ github.action_path }}/scripts/play_store_proxy.py" -- \
|
|
246
|
+
./gradlew "${ANDROID_GRADLE_MODULE}:bundleRelease" \
|
|
234
247
|
--init-script "${{ github.action_path }}/scripts/version_override.init.gradle" \
|
|
235
248
|
--console=plain --stacktrace
|
|
236
249
|
|
|
@@ -32,6 +32,8 @@ import sys
|
|
|
32
32
|
import tempfile
|
|
33
33
|
from pathlib import Path
|
|
34
34
|
|
|
35
|
+
from play_store_proxy import environment
|
|
36
|
+
|
|
35
37
|
# Exact pin, not a range: this job holds the upload keystore and the Play
|
|
36
38
|
# service account, so no floating third-party code runs in it. Bump on purpose.
|
|
37
39
|
FIREBASE_TOOLS = "firebase-tools@15.28.2"
|
|
@@ -97,9 +99,10 @@ def _default_run(cmd: list[str]) -> int:
|
|
|
97
99
|
# A scratch cwd: the Crashlytics buildtools drop a .crashlytics/ directory
|
|
98
100
|
# (dump_syms.bin, ~4 MB) into the working directory, which must not be the
|
|
99
101
|
# consumer's checkout.
|
|
100
|
-
|
|
102
|
+
env = environment()
|
|
101
103
|
try:
|
|
102
|
-
|
|
104
|
+
with tempfile.TemporaryDirectory(prefix="crashlytics-upload-") as workdir:
|
|
105
|
+
return subprocess.run(cmd, check=False, cwd=workdir, env=env).returncode
|
|
103
106
|
except FileNotFoundError:
|
|
104
107
|
print(
|
|
105
108
|
"::error::npx is not on PATH, so the Firebase CLI cannot run. "
|
|
@@ -166,7 +169,7 @@ def main(argv: list[str] | None = None, environ: dict[str, str] | None = None, r
|
|
|
166
169
|
print(
|
|
167
170
|
f"::error::Crashlytics symbol upload failed (exit {rc}). The symbols are "
|
|
168
171
|
"retained as the android-symbols artifact; after fixing the cause, upload "
|
|
169
|
-
f"them
|
|
172
|
+
f"them through the configured proxy after restoring its transport."
|
|
170
173
|
)
|
|
171
174
|
return 1
|
|
172
175
|
print(f"Uploaded Dart symbols to Crashlytics app {app_id}.")
|
|
@@ -9,6 +9,7 @@ import os
|
|
|
9
9
|
from pathlib import Path
|
|
10
10
|
|
|
11
11
|
import google.auth.transport.requests
|
|
12
|
+
from google.auth.exceptions import TransportError
|
|
12
13
|
import requests
|
|
13
14
|
from google.oauth2 import service_account
|
|
14
15
|
|
|
@@ -71,5 +72,5 @@ def main() -> None:
|
|
|
71
72
|
if __name__ == "__main__":
|
|
72
73
|
try:
|
|
73
74
|
main()
|
|
74
|
-
except requests.RequestException:
|
|
75
|
+
except (requests.RequestException, TransportError):
|
|
75
76
|
raise SystemExit("Google Play preflight failed through the assigned proxy") from None
|
|
@@ -1,13 +1,29 @@
|
|
|
1
|
-
"""Google CI transport uses only the account-pinned proxy, including OAuth exchanges.
|
|
1
|
+
"""Google CI transport uses only the account-pinned proxy, including OAuth exchanges.
|
|
2
|
+
|
|
3
|
+
Three entry points, all from the one validated ``GOOGLE_STORE_PROXY_URL``:
|
|
4
|
+
|
|
5
|
+
* ``python3 play_store_proxy.py`` refuses a missing or malformed proxy.
|
|
6
|
+
* ``python3 play_store_proxy.py -- <command …>`` runs a build tool with the explicit
|
|
7
|
+
proxy in both spellings and both ``NO_PROXY`` spellings cleared. Setting the pairs
|
|
8
|
+
as YAML ``env`` keys is not an option: GitHub compares env keys case-insensitively
|
|
9
|
+
and rejects the workflow or action file as invalid.
|
|
10
|
+
* ``python3 play_store_proxy.py --github-env`` clears the lower-case ambient bypass
|
|
11
|
+
(``no_proxy``) for the rest of the job, so a JavaScript upload step that can only
|
|
12
|
+
carry upper-case ``env`` keys cannot be bypassed by a runner's ambient value.
|
|
13
|
+
"""
|
|
2
14
|
from __future__ import annotations
|
|
3
15
|
|
|
4
16
|
import os
|
|
17
|
+
import subprocess
|
|
18
|
+
import sys
|
|
5
19
|
from urllib.parse import urlsplit
|
|
6
20
|
|
|
7
21
|
import requests
|
|
8
22
|
|
|
23
|
+
LOWERCASE_BYPASS = ("no_proxy",)
|
|
9
24
|
|
|
10
|
-
|
|
25
|
+
|
|
26
|
+
def proxy_url() -> str:
|
|
11
27
|
value = os.environ.get("GOOGLE_STORE_PROXY_URL", "").strip()
|
|
12
28
|
try:
|
|
13
29
|
parsed = urlsplit(value)
|
|
@@ -16,7 +32,54 @@ def session() -> requests.Session:
|
|
|
16
32
|
valid = False
|
|
17
33
|
if not valid:
|
|
18
34
|
raise SystemExit("GOOGLE_STORE_PROXY_URL must contain the account-pinned proxy; direct requests are refused")
|
|
35
|
+
return value
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
def environment() -> dict[str, str]:
|
|
39
|
+
value = proxy_url()
|
|
40
|
+
return {**os.environ, "HTTP_PROXY": value, "HTTPS_PROXY": value,
|
|
41
|
+
"http_proxy": value, "https_proxy": value, "NO_PROXY": "", "no_proxy": ""}
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
def session() -> requests.Session:
|
|
45
|
+
value = proxy_url()
|
|
19
46
|
client = requests.Session()
|
|
20
47
|
client.trust_env = False
|
|
21
48
|
client.proxies = {"http": value, "https": value}
|
|
22
49
|
return client
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
def run(argv: list[str]) -> int:
|
|
53
|
+
"""Run ``argv`` with the explicit proxy environment; the parent process is untouched."""
|
|
54
|
+
if not argv:
|
|
55
|
+
raise SystemExit("usage: play_store_proxy.py -- <command …>")
|
|
56
|
+
env = environment()
|
|
57
|
+
return subprocess.run(argv, env=env, check=False).returncode
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def clear_github_env_bypass(path: str | None = None) -> list[str]:
|
|
61
|
+
"""Append empty lower-case bypass assignments to ``$GITHUB_ENV`` for later steps."""
|
|
62
|
+
proxy_url()
|
|
63
|
+
target = path or os.environ.get("GITHUB_ENV", "")
|
|
64
|
+
if not target:
|
|
65
|
+
raise SystemExit("GITHUB_ENV is not set; run this inside a GitHub Actions step")
|
|
66
|
+
lines = [f"{name}=" for name in LOWERCASE_BYPASS]
|
|
67
|
+
with open(target, "a", encoding="utf-8") as handle:
|
|
68
|
+
handle.write("".join(line + "\n" for line in lines))
|
|
69
|
+
return lines
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
def main(args: list[str]) -> int:
|
|
73
|
+
if args[:1] == ["--github-env"]:
|
|
74
|
+
clear_github_env_bypass()
|
|
75
|
+
return 0
|
|
76
|
+
if args[:1] == ["--"]:
|
|
77
|
+
return run(args[1:])
|
|
78
|
+
if args:
|
|
79
|
+
raise SystemExit("usage: play_store_proxy.py [--github-env | -- <command …>]")
|
|
80
|
+
proxy_url()
|
|
81
|
+
return 0
|
|
82
|
+
|
|
83
|
+
|
|
84
|
+
if __name__ == "__main__":
|
|
85
|
+
sys.exit(main(sys.argv[1:]))
|
|
@@ -10,10 +10,12 @@ the bottom, next to the other Flutter obfuscation wiring tests.
|
|
|
10
10
|
from __future__ import annotations
|
|
11
11
|
|
|
12
12
|
import io
|
|
13
|
+
import os
|
|
13
14
|
import re
|
|
14
15
|
import sys
|
|
15
16
|
import tempfile
|
|
16
17
|
import unittest
|
|
18
|
+
from unittest import mock
|
|
17
19
|
from contextlib import redirect_stdout
|
|
18
20
|
from pathlib import Path
|
|
19
21
|
|
|
@@ -140,7 +142,8 @@ class MainTest(unittest.TestCase):
|
|
|
140
142
|
|
|
141
143
|
def test_a_missing_npx_is_a_named_error_not_a_traceback(self) -> None:
|
|
142
144
|
out = io.StringIO()
|
|
143
|
-
with redirect_stdout(out)
|
|
145
|
+
with redirect_stdout(out), mock.patch.dict(os.environ, {
|
|
146
|
+
"GOOGLE_STORE_PROXY_URL": "http://assigned.proxy.test:1234"}):
|
|
144
147
|
rc = cs._default_run(["gowalk-cicd-no-such-binary-xyz"])
|
|
145
148
|
self.assertNotEqual(rc, 0)
|
|
146
149
|
self.assertIn("::error::", out.getvalue())
|
|
@@ -17,6 +17,16 @@ class PlayProxyTests(unittest.TestCase):
|
|
|
17
17
|
play_store_proxy.session()
|
|
18
18
|
session.assert_not_called()
|
|
19
19
|
|
|
20
|
+
def test_cli_environment_is_explicit_without_mutating_the_parent(self):
|
|
21
|
+
value = "http://assigned.proxy.test:1234"
|
|
22
|
+
with mock.patch.dict(os.environ, {"GOOGLE_STORE_PROXY_URL": value, "NO_PROXY": "*"}):
|
|
23
|
+
child = play_store_proxy.environment()
|
|
24
|
+
self.assertEqual(child["HTTPS_PROXY"], value)
|
|
25
|
+
self.assertEqual(child["http_proxy"], value)
|
|
26
|
+
self.assertEqual(child["NO_PROXY"], "")
|
|
27
|
+
self.assertEqual(child["no_proxy"], "")
|
|
28
|
+
self.assertEqual(os.environ["NO_PROXY"], "*")
|
|
29
|
+
|
|
20
30
|
def test_ambient_proxy_and_no_proxy_cannot_override_account_exit(self):
|
|
21
31
|
value = "http://assigned.proxy.test:1234"
|
|
22
32
|
with mock.patch.dict(os.environ, {"GOOGLE_STORE_PROXY_URL": value,
|
|
@@ -25,6 +35,47 @@ class PlayProxyTests(unittest.TestCase):
|
|
|
25
35
|
self.assertFalse(client.trust_env)
|
|
26
36
|
self.assertEqual(client.proxies, {"http": value, "https": value})
|
|
27
37
|
|
|
38
|
+
def test_exec_form_gives_the_child_both_spellings_and_no_bypass(self):
|
|
39
|
+
value = "http://assigned.proxy.test:1234"
|
|
40
|
+
probe = ("import os, json; print(json.dumps({k: os.environ.get(k) for k in "
|
|
41
|
+
"['HTTPS_PROXY', 'https_proxy', 'HTTP_PROXY', 'http_proxy', 'NO_PROXY', 'no_proxy']}))")
|
|
42
|
+
with mock.patch.dict(os.environ, {"GOOGLE_STORE_PROXY_URL": value, "no_proxy": "*", "NO_PROXY": "*"}), \
|
|
43
|
+
mock.patch.object(play_store_proxy.subprocess, "run") as run:
|
|
44
|
+
run.return_value.returncode = 0
|
|
45
|
+
self.assertEqual(play_store_proxy.run([sys.executable, "-c", probe]), 0)
|
|
46
|
+
child = run.call_args.kwargs["env"]
|
|
47
|
+
self.assertEqual(child["https_proxy"], value)
|
|
48
|
+
self.assertEqual(child["HTTP_PROXY"], value)
|
|
49
|
+
self.assertEqual(child["no_proxy"], "")
|
|
50
|
+
self.assertEqual(child["NO_PROXY"], "")
|
|
51
|
+
self.assertEqual(os.environ["no_proxy"], "*")
|
|
52
|
+
|
|
53
|
+
def test_exec_form_refuses_without_a_proxy_before_running_anything(self):
|
|
54
|
+
with mock.patch.dict(os.environ, {"GOOGLE_STORE_PROXY_URL": ""}), \
|
|
55
|
+
mock.patch.object(play_store_proxy.subprocess, "run") as run:
|
|
56
|
+
with self.assertRaises(SystemExit):
|
|
57
|
+
play_store_proxy.run(["true"])
|
|
58
|
+
run.assert_not_called()
|
|
59
|
+
|
|
60
|
+
def test_github_env_mode_clears_only_the_lowercase_bypass_for_later_steps(self):
|
|
61
|
+
import tempfile
|
|
62
|
+
value = "http://assigned.proxy.test:1234"
|
|
63
|
+
with tempfile.TemporaryDirectory() as tmp:
|
|
64
|
+
target = os.path.join(tmp, "github.env")
|
|
65
|
+
with mock.patch.dict(os.environ, {"GOOGLE_STORE_PROXY_URL": value, "GITHUB_ENV": target, "no_proxy": "*"}):
|
|
66
|
+
self.assertEqual(play_store_proxy.clear_github_env_bypass(), ["no_proxy="])
|
|
67
|
+
self.assertEqual(os.environ["no_proxy"], "*")
|
|
68
|
+
with open(target, encoding="utf-8") as handle:
|
|
69
|
+
self.assertEqual(handle.read(), "no_proxy=\n")
|
|
70
|
+
|
|
71
|
+
def test_github_env_mode_refuses_without_a_proxy_or_target(self):
|
|
72
|
+
with mock.patch.dict(os.environ, {"GOOGLE_STORE_PROXY_URL": "", "GITHUB_ENV": "/nonexistent"}):
|
|
73
|
+
with self.assertRaises(SystemExit):
|
|
74
|
+
play_store_proxy.clear_github_env_bypass()
|
|
75
|
+
with mock.patch.dict(os.environ, {"GOOGLE_STORE_PROXY_URL": "http://assigned.proxy.test:1234", "GITHUB_ENV": ""}):
|
|
76
|
+
with self.assertRaises(SystemExit):
|
|
77
|
+
play_store_proxy.clear_github_env_bypass()
|
|
78
|
+
|
|
28
79
|
|
|
29
80
|
if __name__ == "__main__":
|
|
30
81
|
unittest.main()
|
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.60
|
package/package.json
CHANGED
package/templates/deploy.yml
CHANGED
|
@@ -453,6 +453,14 @@ jobs:
|
|
|
453
453
|
# Note this also means nothing else should be creating edits against these
|
|
454
454
|
# packages while a deploy runs — an external poller that opens and deletes an
|
|
455
455
|
# edit will invalidate the one this step is holding.
|
|
456
|
+
# The upload action is JavaScript, so its proxy can only be the step's own
|
|
457
|
+
# upper-case env keys below. Node clients fall back to a lower-case no_proxy
|
|
458
|
+
# when NO_PROXY is empty, and GitHub refuses both spellings in one env map, so
|
|
459
|
+
# the ambient lower-case bypass is cleared through GITHUB_ENV instead.
|
|
460
|
+
- name: Clear ambient lower-case proxy bypass
|
|
461
|
+
if: ${{ steps.mode.outputs.mode == 'local' && steps.play.outputs.ready == 'true' }}
|
|
462
|
+
shell: bash
|
|
463
|
+
run: python3 .github/actions/android-app/scripts/play_store_proxy.py --github-env
|
|
456
464
|
- name: Upload Android release to Google Play
|
|
457
465
|
id: play-upload
|
|
458
466
|
continue-on-error: true
|
|
@@ -461,8 +469,7 @@ jobs:
|
|
|
461
469
|
env:
|
|
462
470
|
HTTPS_PROXY: ${{ secrets.GOOGLE_STORE_PROXY_URL }}
|
|
463
471
|
HTTP_PROXY: ${{ secrets.GOOGLE_STORE_PROXY_URL }}
|
|
464
|
-
NO_PROXY: ''
|
|
465
|
-
no_proxy: ''
|
|
472
|
+
NO_PROXY: '' # one spelling only: GitHub rejects env keys that differ by case
|
|
466
473
|
with:
|
|
467
474
|
serviceAccountJson: ${{ steps.android.outputs.play-service-account }}
|
|
468
475
|
packageName: ${{ steps.android.outputs.package-name }}
|
|
@@ -490,8 +497,7 @@ jobs:
|
|
|
490
497
|
env:
|
|
491
498
|
HTTPS_PROXY: ${{ secrets.GOOGLE_STORE_PROXY_URL }}
|
|
492
499
|
HTTP_PROXY: ${{ secrets.GOOGLE_STORE_PROXY_URL }}
|
|
493
|
-
NO_PROXY: ''
|
|
494
|
-
no_proxy: ''
|
|
500
|
+
NO_PROXY: '' # one spelling only: GitHub rejects env keys that differ by case
|
|
495
501
|
with:
|
|
496
502
|
serviceAccountJson: ${{ steps.android.outputs.play-service-account }}
|
|
497
503
|
packageName: ${{ steps.android.outputs.package-name }}
|