gowalk-cicd 1.0.96 → 1.0.98
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +10 -0
- package/README.md +11 -0
- package/action/.daemux-version +1 -1
- package/android-action/.daemux-version +1 -1
- package/android-action/action.yml +4 -1
- package/android-action/flutter-setup/action.yml +26 -0
- package/android-action/scripts/android_build_diagnostics.py +80 -0
- package/android-action/scripts/flutter_archive_cache.py +47 -0
- package/android-action/scripts/flutter_download.py +12 -9
- package/android-action/scripts/flutter_manifest.py +9 -4
- package/android-action/scripts/test_android_build_diagnostics.py +108 -0
- package/android-action/scripts/test_flutter_archive_cache.py +102 -0
- package/android-action/scripts/test_google_bootstrap_wiring.py +13 -4
- package/backend-action/.daemux-version +1 -1
- package/package.json +1 -1
package/CLAUDE.md
CHANGED
|
@@ -165,6 +165,10 @@ node /path/to/gowalk-cicd/bin/cli.mjs
|
|
|
165
165
|
`flutter_download_failed` emits fixed transport codes and retained-byte counts, never curl stderr.
|
|
166
166
|
Archive progress emits retained byte counts every minute and safe results after each attempt;
|
|
167
167
|
the transfer deadline leaves room for a failure receipt before a 30-minute validation job ends.
|
|
168
|
+
With SDK caching enabled, failed archive transfers preserve public bytes in a job-owned,
|
|
169
|
+
digest-scoped directory. Separate optional cache restore/save steps retain them between runs;
|
|
170
|
+
they never expose partial bytes as an installed SDK. Successful or corrupt completed archives
|
|
171
|
+
are removed from that partial cache. Full SDK cache identities and provider transport stay unchanged.
|
|
168
172
|
- Web Release selects bundled renderer resources with `--no-web-resources-cdn` inside
|
|
169
173
|
the existing proxied build command. This is not a font or integration offline guarantee;
|
|
170
174
|
preserve app-authored font declarations, bootstrap configuration and other build inputs.
|
|
@@ -223,6 +227,12 @@ node /path/to/gowalk-cicd/bin/cli.mjs
|
|
|
223
227
|
is the only place that decides which; every downstream step branches on the
|
|
224
228
|
`project_kind` output rather than re-sniffing the repo. Flutter wins the tie
|
|
225
229
|
because a Flutter app also carries `android/settings.gradle`.
|
|
230
|
+
- Both Android release compilers run through `android_build_diagnostics.py` inside the
|
|
231
|
+
existing proxy wrapper. It retains at most 64 KiB of private child output and publishes
|
|
232
|
+
only fixed signal names, phase and exit status in `android_build_failed`. An empty signal
|
|
233
|
+
list is unclassified, never a guessed cause; matching signals are observations, not a retry
|
|
234
|
+
decision. Minute progress notices replace raw compiler output. Preserve the actual exit,
|
|
235
|
+
command arguments, proxy environment, signing and symbol checks.
|
|
226
236
|
- iOS plist stamping reads complete JSON build settings through `resolve_info_plist.py`,
|
|
227
237
|
preserving Xcode's exit status and bounded sanitized stderr. Select exactly the configured
|
|
228
238
|
bundle's application target and resolve paths from that target's source root; never take
|
package/README.md
CHANGED
|
@@ -314,6 +314,12 @@ the situation and whose **message** is one line of JSON carrying a versioned
|
|
|
314
314
|
| `source_maintenance_pending` | `gowalk-cicd/source-maintenance-pending.v1` | `repo`, `base_sha`, `head_sha`, `branch`, `run_id` |
|
|
315
315
|
| `apple_build_upload_failed` (error) | `gowalk-cicd/apple-build-upload-failed.v2` | `category`, `stage`, `receipt_available`, receipt identifiers (`cfBundleVersion`, `upload_id`, `file_id`, `state`, `provider_status`, `provider_code`, `provider_pointer`), `action` |
|
|
316
316
|
| `flutter_download_failed` (error) | `gowalk-cicd/flutter-download-failed.v1` | `phase`, `curl_exit`, `http_status`, `proxy_status`, `attempts`, `retained_bytes` |
|
|
317
|
+
| `android_build_failed` (error) | `gowalk-cicd/android-build-failed.v1` | `phase`, `exit_code`, `signals` (fixed observed patterns; empty means unclassified) |
|
|
318
|
+
|
|
319
|
+
Android release compilers retain only a bounded private output window. Their failure signals
|
|
320
|
+
describe observed error patterns, not retry permission or a unique root cause. Silent builds
|
|
321
|
+
emit `android_build_progress` notices (`gowalk-cicd/android-build-progress.v1`) every minute
|
|
322
|
+
with `phase` and `elapsed_seconds`; successful compiler output is not republished.
|
|
317
323
|
|
|
318
324
|
SDK transfers also emit `flutter_download_progress` notices with schema
|
|
319
325
|
`gowalk-cicd/flutter-download-progress.v1`. Each carries `phase`, `event`, `elapsed_ms`
|
|
@@ -321,6 +327,11 @@ and `retained_bytes`; `attempt_finished` adds `attempt`, `curl_exit`, `http_stat
|
|
|
321
327
|
and `proxy_status`. In-progress notices arrive each minute; elapsed time belongs to
|
|
322
328
|
the current attempt. Archives have a 20-minute total transfer budget so a failure
|
|
323
329
|
receipt precedes a 30-minute validation job's deadline. Progress is not a success receipt.
|
|
330
|
+
When SDK caching is enabled, an exhausted transfer preserves its partial public archive for
|
|
331
|
+
the next run under a separate release-specific cache key. Resumption still uses the selected
|
|
332
|
+
official manifest digest and the configured provider proxy; only a verified complete archive
|
|
333
|
+
can reach SDK extraction. Partial-cache restore uses the existing one-minute cache segment
|
|
334
|
+
bound. Optional cache operations do not extend the transfer deadline or turn a failed installation green.
|
|
324
335
|
|
|
325
336
|
Add a field by bumping the schema version; never change the meaning of an
|
|
326
337
|
existing one.
|
package/action/.daemux-version
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.98
|
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.98
|
|
@@ -178,7 +178,9 @@ runs:
|
|
|
178
178
|
*) echo "::error::dart-defines entry '$define' is not KEY=VALUE"; exit 1 ;;
|
|
179
179
|
esac
|
|
180
180
|
done
|
|
181
|
-
python3 "${{ github.action_path }}/scripts/play_store_proxy.py" --
|
|
181
|
+
python3 "${{ github.action_path }}/scripts/play_store_proxy.py" -- \
|
|
182
|
+
python3 "${{ github.action_path }}/scripts/android_build_diagnostics.py" --phase flutter_bundle -- \
|
|
183
|
+
flutter "${args[@]}"
|
|
182
184
|
if ! ls "$DART_SYMBOLS_DIR"/*.symbols >/dev/null 2>&1; then
|
|
183
185
|
echo "::error::flutter build wrote no Dart symbol files to $DART_SYMBOLS_DIR;" \
|
|
184
186
|
"a crash from this obfuscated build could never be read. Refusing to ship it."
|
|
@@ -267,6 +269,7 @@ runs:
|
|
|
267
269
|
run: |
|
|
268
270
|
chmod +x ./gradlew
|
|
269
271
|
python3 "${{ github.action_path }}/scripts/play_store_proxy.py" -- \
|
|
272
|
+
python3 "${{ github.action_path }}/scripts/android_build_diagnostics.py" --phase gradle_bundle -- \
|
|
270
273
|
./gradlew "${ANDROID_GRADLE_MODULE}:bundleRelease" \
|
|
271
274
|
--init-script "${{ github.action_path }}/scripts/version_override.init.gradle" \
|
|
272
275
|
--console=plain --stacktrace
|
|
@@ -138,6 +138,18 @@ runs:
|
|
|
138
138
|
with:
|
|
139
139
|
path: ${{ steps.flutter-action.outputs.PUB-CACHE-PATH }}
|
|
140
140
|
key: ${{ steps.flutter-action.outputs.PUB-CACHE-KEY }}-${{ hashFiles('**/pubspec.lock') }}
|
|
141
|
+
- name: Restore interrupted Flutter archive
|
|
142
|
+
if: ${{ inputs.cache == 'true' && inputs.dry-run != 'true' && steps.cache-flutter.outputs.cache-hit != 'true' }}
|
|
143
|
+
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25
|
|
144
|
+
continue-on-error: true
|
|
145
|
+
env:
|
|
146
|
+
NODE_OPTIONS: --require "${{ steps.scope.outputs.preloader }}"
|
|
147
|
+
SEGMENT_DOWNLOAD_TIMEOUT_MINS: '1'
|
|
148
|
+
with:
|
|
149
|
+
path: ${{ runner.temp }}/gowalk-flutter-archives
|
|
150
|
+
key: >-
|
|
151
|
+
flutter-archive-v1-${{ steps.flutter-action.outputs.CACHE-KEY }}-${{ github.run_id }}-${{ github.run_attempt }}
|
|
152
|
+
restore-keys: flutter-archive-v1-${{ steps.flutter-action.outputs.CACHE-KEY }}-
|
|
141
153
|
- name: Install or reuse the resolved Flutter SDK
|
|
142
154
|
id: install
|
|
143
155
|
if: ${{ inputs.dry-run != 'true' }}
|
|
@@ -145,6 +157,8 @@ runs:
|
|
|
145
157
|
env:
|
|
146
158
|
BASH_ENV: ${{ steps.transport.outputs.manifest_env }}
|
|
147
159
|
GOWALK_FLUTTER_MANIFEST_HELPER: ${{ steps.transport.outputs.manifest_helper }}
|
|
160
|
+
GOWALK_FLUTTER_ARCHIVE_CACHE: >-
|
|
161
|
+
${{ inputs.cache == 'true' && format('{0}/gowalk-flutter-archives', runner.temp) || '' }}
|
|
148
162
|
with:
|
|
149
163
|
channel: ${{ steps.flutter-action.outputs.CHANNEL }}
|
|
150
164
|
flutter-version: ${{ steps.flutter-action.outputs.VERSION }}
|
|
@@ -158,6 +172,18 @@ runs:
|
|
|
158
172
|
pub-cache-path: ${{ steps.flutter-action.outputs.PUB-CACHE-PATH }}
|
|
159
173
|
dry-run: 'false'
|
|
160
174
|
git-source: ${{ inputs.git-source }}
|
|
175
|
+
- name: Preserve interrupted Flutter archive
|
|
176
|
+
if: >-
|
|
177
|
+
${{ !cancelled() && inputs.cache == 'true' && steps.install.outcome == 'failure'
|
|
178
|
+
&& steps.scope.outcome == 'success' }}
|
|
179
|
+
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25
|
|
180
|
+
continue-on-error: true
|
|
181
|
+
env:
|
|
182
|
+
NODE_OPTIONS: --require "${{ steps.scope.outputs.preloader }}"
|
|
183
|
+
with:
|
|
184
|
+
path: ${{ runner.temp }}/gowalk-flutter-archives
|
|
185
|
+
key: >-
|
|
186
|
+
flutter-archive-v1-${{ steps.flutter-action.outputs.CACHE-KEY }}-${{ github.run_id }}-${{ github.run_attempt }}
|
|
161
187
|
# A later app failure must not discard a successfully installed, expensive SDK.
|
|
162
188
|
- name: Save installed Flutter SDK
|
|
163
189
|
if: >-
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
"""Run one Android compiler with bounded private output and fixed failure signals."""
|
|
2
|
+
import argparse
|
|
3
|
+
import json
|
|
4
|
+
import subprocess
|
|
5
|
+
import sys
|
|
6
|
+
import threading
|
|
7
|
+
import time
|
|
8
|
+
|
|
9
|
+
LIMIT = 65536
|
|
10
|
+
SIGNALS = {
|
|
11
|
+
"tls_verification": ("PKIX path building failed", "CERTIFICATE_VERIFY_FAILED", "certificate verify failed"),
|
|
12
|
+
"dependency_resolution": ("Could not resolve all files", "Could not resolve all artifacts", "Could not GET '"),
|
|
13
|
+
"connection_timeout": ("Connect timed out", "Read timed out", "Connection timed out"),
|
|
14
|
+
"kotlin_compilation": ("Compilation error. See log for more details", "Kotlin compilation failed"),
|
|
15
|
+
"dart_compilation": ("Target kernel_snapshot_program failed", "Target kernel_snapshot failed"),
|
|
16
|
+
"android_resources": ("Android resource linking failed", "Android resource compilation failed"),
|
|
17
|
+
"r8_missing_classes": ("Missing classes detected while running R8",),
|
|
18
|
+
"duplicate_classes": ("Duplicate class ",),
|
|
19
|
+
"manifest_merge": ("Manifest merger failed",),
|
|
20
|
+
"signing_configuration": ("Keystore was tampered with", "Keystore file ", "Failed to read key "),
|
|
21
|
+
"out_of_memory": ("java.lang.OutOfMemoryError", "Java heap space", "GC overhead limit exceeded"),
|
|
22
|
+
"gradle_daemon_lost": ("Gradle build daemon disappeared unexpectedly",),
|
|
23
|
+
"disk_full": ("No space left on device",),
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
def signals(output: bytes) -> list[str]:
|
|
28
|
+
text = output.decode("utf-8", errors="replace")
|
|
29
|
+
return sorted(name for name, patterns in SIGNALS.items() if any(pattern in text for pattern in patterns))
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
def emit(title: str, level: str, **fields) -> None:
|
|
33
|
+
print(f"::{level} title={title}::" + json.dumps(fields, separators=(",", ":")), flush=True)
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
def drain(stream, tail: bytearray) -> None:
|
|
37
|
+
while chunk := stream.read1(8192):
|
|
38
|
+
tail[:] = (tail + chunk)[-LIMIT:]
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
def run(phase: str, command: list[str], *, interval: float = 60) -> int:
|
|
42
|
+
started, tail = time.monotonic(), bytearray()
|
|
43
|
+
try:
|
|
44
|
+
child = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
|
|
45
|
+
except OSError:
|
|
46
|
+
emit("android_build_failed", "error", schema="gowalk-cicd/android-build-failed.v1",
|
|
47
|
+
phase=phase, exit_code=127, signals=["command_start_failed"])
|
|
48
|
+
return 127
|
|
49
|
+
reader = threading.Thread(target=drain, args=(child.stdout, tail), daemon=True)
|
|
50
|
+
reader.start()
|
|
51
|
+
while True:
|
|
52
|
+
try:
|
|
53
|
+
code = child.wait(timeout=interval)
|
|
54
|
+
break
|
|
55
|
+
except subprocess.TimeoutExpired:
|
|
56
|
+
emit("android_build_progress", "notice", schema="gowalk-cicd/android-build-progress.v1",
|
|
57
|
+
phase=phase, elapsed_seconds=int(time.monotonic() - started))
|
|
58
|
+
# A descendant retaining stdout must not hold a finished compiler's result indefinitely.
|
|
59
|
+
reader.join(timeout=1)
|
|
60
|
+
if not reader.is_alive():
|
|
61
|
+
child.stdout.close()
|
|
62
|
+
if code:
|
|
63
|
+
emit("android_build_failed", "error", schema="gowalk-cicd/android-build-failed.v1",
|
|
64
|
+
phase=phase, exit_code=code, signals=signals(bytes(tail)))
|
|
65
|
+
return code if code >= 0 else 128 - code
|
|
66
|
+
|
|
67
|
+
|
|
68
|
+
def main(argv: list[str]) -> int:
|
|
69
|
+
parser = argparse.ArgumentParser(description=__doc__)
|
|
70
|
+
parser.add_argument("--phase", required=True, choices=("flutter_bundle", "gradle_bundle"))
|
|
71
|
+
parser.add_argument("command", nargs=argparse.REMAINDER)
|
|
72
|
+
args = parser.parse_args(argv)
|
|
73
|
+
command = args.command[1:] if args.command[:1] == ["--"] else args.command
|
|
74
|
+
if not command:
|
|
75
|
+
parser.error("a build command is required")
|
|
76
|
+
return run(args.phase, command)
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+
if __name__ == "__main__":
|
|
80
|
+
sys.exit(main(sys.argv[1:]))
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
"""Keep only public SDK archive bytes in the job's optional, digest-scoped download cache."""
|
|
2
|
+
from contextlib import contextmanager
|
|
3
|
+
import os
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
import tempfile
|
|
6
|
+
|
|
7
|
+
import flutter_download
|
|
8
|
+
|
|
9
|
+
|
|
10
|
+
def cached_directory() -> Path:
|
|
11
|
+
root = Path(os.environ["GOWALK_FLUTTER_ARCHIVE_CACHE"])
|
|
12
|
+
temporary = Path(os.environ["RUNNER_TEMP"]).resolve(strict=True)
|
|
13
|
+
if root.is_symlink() or root.parent.resolve(strict=True) != temporary:
|
|
14
|
+
raise ValueError("flutter_archive_cache_scope_invalid")
|
|
15
|
+
root.mkdir(mode=0o700, exist_ok=True)
|
|
16
|
+
directory = root / flutter_download.archive_digest()
|
|
17
|
+
if directory.is_symlink():
|
|
18
|
+
raise ValueError("flutter_archive_cache_scope_invalid")
|
|
19
|
+
directory.mkdir(mode=0o700, exist_ok=True)
|
|
20
|
+
destination = directory / "download"
|
|
21
|
+
if destination.is_symlink() or (destination.exists() and not destination.is_file()):
|
|
22
|
+
raise ValueError("flutter_archive_cache_scope_invalid")
|
|
23
|
+
if destination.exists() and destination.stat().st_size > flutter_download.MAX_ARCHIVE_BYTES:
|
|
24
|
+
raise ValueError("flutter_archive_size_exceeded")
|
|
25
|
+
return directory
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
@contextmanager
|
|
29
|
+
def workspace(kind: str):
|
|
30
|
+
if kind != "archive" or not os.environ.get("GOWALK_FLUTTER_ARCHIVE_CACHE"):
|
|
31
|
+
with tempfile.TemporaryDirectory(prefix="flutter-manifest-", dir=os.environ["RUNNER_TEMP"]) as scratch:
|
|
32
|
+
yield Path(scratch)
|
|
33
|
+
return
|
|
34
|
+
directory = cached_directory()
|
|
35
|
+
try:
|
|
36
|
+
yield directory
|
|
37
|
+
except ValueError as error:
|
|
38
|
+
if str(error) == "flutter_archive_checksum_mismatch":
|
|
39
|
+
(directory / "download").unlink(missing_ok=True)
|
|
40
|
+
raise
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
def reuse_complete(destination: Path, kind: str) -> bool:
|
|
44
|
+
if kind != "archive" or not destination.stat().st_size or not flutter_download.archive_matches(destination):
|
|
45
|
+
return False
|
|
46
|
+
flutter_download.publish(destination, kind)
|
|
47
|
+
return True
|
|
@@ -63,18 +63,21 @@ def progress(destination: Path, started: float, event: str, **fields) -> None:
|
|
|
63
63
|
file=sys.stderr, flush=True)
|
|
64
64
|
|
|
65
65
|
|
|
66
|
+
def archive_matches(destination: Path) -> bool:
|
|
67
|
+
if destination.stat().st_size > MAX_ARCHIVE_BYTES:
|
|
68
|
+
raise ValueError("flutter_archive_size_exceeded")
|
|
69
|
+
digest = hashlib.sha256()
|
|
70
|
+
with destination.open("rb") as source:
|
|
71
|
+
for chunk in iter(lambda: source.read(1024 * 1024), b""):
|
|
72
|
+
digest.update(chunk)
|
|
73
|
+
return digest.hexdigest() == archive_digest()
|
|
74
|
+
|
|
75
|
+
|
|
66
76
|
def publish(destination: Path, kind: str) -> None:
|
|
67
77
|
if kind == "manifest":
|
|
68
78
|
json.loads(destination.read_bytes())
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
raise ValueError("flutter_archive_size_exceeded")
|
|
72
|
-
digest = hashlib.sha256()
|
|
73
|
-
with destination.open("rb") as source:
|
|
74
|
-
for chunk in iter(lambda: source.read(1024 * 1024), b""):
|
|
75
|
-
digest.update(chunk)
|
|
76
|
-
if digest.hexdigest() != archive_digest():
|
|
77
|
-
raise ValueError("flutter_archive_checksum_mismatch")
|
|
79
|
+
elif not archive_matches(destination):
|
|
80
|
+
raise ValueError("flutter_archive_checksum_mismatch")
|
|
78
81
|
with destination.open("rb") as source:
|
|
79
82
|
shutil.copyfileobj(source, sys.stdout.buffer, length=1024 * 1024)
|
|
80
83
|
|
|
@@ -5,11 +5,11 @@ import os
|
|
|
5
5
|
from pathlib import Path
|
|
6
6
|
import re
|
|
7
7
|
import sys
|
|
8
|
-
import tempfile
|
|
9
8
|
import time
|
|
10
9
|
from urllib.parse import urlsplit
|
|
11
10
|
|
|
12
11
|
import flutter_download
|
|
12
|
+
import flutter_archive_cache
|
|
13
13
|
|
|
14
14
|
RETRY_CODES = {5, 6, 7, 18, 28, 35, 52, 55, 56, 92, 95}
|
|
15
15
|
RETRY_HTTP = {408, 429, 500, 502, 503, 504}
|
|
@@ -24,8 +24,10 @@ def download(arguments: list[str], directory: Path) -> int:
|
|
|
24
24
|
child_env = {**os.environ, "NO_PROXY": "", "no_proxy": ""}
|
|
25
25
|
deadline, code, http, proxy_http = time.monotonic() + (60 if kind == "manifest" else 1200), 1, 0, 0
|
|
26
26
|
destination = directory / "download"
|
|
27
|
-
destination.touch(mode=0o600, exist_ok=
|
|
27
|
+
destination.touch(mode=0o600, exist_ok=True)
|
|
28
28
|
destination.chmod(0o600)
|
|
29
|
+
if flutter_archive_cache.reuse_complete(destination, kind):
|
|
30
|
+
return 0
|
|
29
31
|
attempted = 0
|
|
30
32
|
for attempt in range(1, 5):
|
|
31
33
|
remaining = deadline - time.monotonic()
|
|
@@ -74,8 +76,11 @@ def request_kind(arguments):
|
|
|
74
76
|
|
|
75
77
|
def main() -> int:
|
|
76
78
|
try:
|
|
77
|
-
with
|
|
78
|
-
|
|
79
|
+
with flutter_archive_cache.workspace(request_kind(sys.argv[1:])) as scratch:
|
|
80
|
+
result = download(sys.argv[1:], scratch)
|
|
81
|
+
if result == 0:
|
|
82
|
+
(scratch / "download").unlink(missing_ok=True)
|
|
83
|
+
return result
|
|
79
84
|
except (ValueError, OSError, KeyError) as error:
|
|
80
85
|
safe = {"flutter_archive_digest_required", "flutter_archive_size_exceeded", "flutter_archive_checksum_mismatch"}
|
|
81
86
|
reason = str(error) if str(error) in safe else "flutter_manifest_invalid_request_or_response"
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
"""Exercise real child failures without publishing child output or altering its environment."""
|
|
2
|
+
import contextlib
|
|
3
|
+
import io
|
|
4
|
+
import json
|
|
5
|
+
import os
|
|
6
|
+
from pathlib import Path
|
|
7
|
+
import subprocess
|
|
8
|
+
import sys
|
|
9
|
+
import tempfile
|
|
10
|
+
import unittest
|
|
11
|
+
from unittest import mock
|
|
12
|
+
|
|
13
|
+
import android_build_diagnostics as diagnostics
|
|
14
|
+
import yaml
|
|
15
|
+
from test_jvm_proxy_relay import upstream
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
class BuildDiagnosticsTests(unittest.TestCase):
|
|
19
|
+
def invoke(self, script, **kwargs):
|
|
20
|
+
output = io.StringIO()
|
|
21
|
+
with contextlib.redirect_stdout(output):
|
|
22
|
+
code = diagnostics.run("flutter_bundle", [sys.executable, "-c", script], **kwargs)
|
|
23
|
+
rows = [json.loads(line.split("::", 2)[2]) for line in output.getvalue().splitlines()]
|
|
24
|
+
return code, rows, output.getvalue()
|
|
25
|
+
|
|
26
|
+
def test_child_failure_keeps_exit_and_only_fixed_signals(self):
|
|
27
|
+
code, rows, output = self.invoke(
|
|
28
|
+
"import sys; print('secret=do-not-publish https://user:password@example.test');"
|
|
29
|
+
"print('Missing classes detected while running R8'); sys.exit(42)"
|
|
30
|
+
)
|
|
31
|
+
self.assertEqual(code, 42)
|
|
32
|
+
self.assertEqual(rows, [{"schema": "gowalk-cicd/android-build-failed.v1", "phase": "flutter_bundle",
|
|
33
|
+
"exit_code": 42, "signals": ["r8_missing_classes"]}])
|
|
34
|
+
self.assertNotIn("password", output)
|
|
35
|
+
self.assertNotIn("do-not-publish", output)
|
|
36
|
+
|
|
37
|
+
def test_unknown_failure_never_guesses_or_echoes_output(self):
|
|
38
|
+
code, rows, output = self.invoke("import sys; print('private provider exception'); sys.exit(1)")
|
|
39
|
+
self.assertEqual(code, 1)
|
|
40
|
+
self.assertEqual(rows[0]["signals"], [])
|
|
41
|
+
self.assertNotIn("private provider", output)
|
|
42
|
+
|
|
43
|
+
def test_success_preserves_environment_and_arguments_without_echoing_them(self):
|
|
44
|
+
with mock.patch.dict(os.environ, {"HTTPS_PROXY": "http://private.invalid:9", "NO_PROXY": ""}):
|
|
45
|
+
code, rows, output = self.invoke(
|
|
46
|
+
"import os; assert os.environ['HTTPS_PROXY']=='http://private.invalid:9';"
|
|
47
|
+
"assert os.environ['NO_PROXY']==''; print('secret successful compiler output')"
|
|
48
|
+
)
|
|
49
|
+
self.assertEqual((code, rows, output), (0, [], ""))
|
|
50
|
+
|
|
51
|
+
def test_large_output_stays_bounded_and_recent_failures_are_retained(self):
|
|
52
|
+
tail = bytearray()
|
|
53
|
+
content = b"secret" * diagnostics.LIMIT + b"Android resource linking failed"
|
|
54
|
+
with io.BufferedReader(io.BytesIO(content)) as stream:
|
|
55
|
+
diagnostics.drain(stream, tail)
|
|
56
|
+
self.assertEqual(len(tail), diagnostics.LIMIT)
|
|
57
|
+
self.assertEqual(diagnostics.signals(tail), ["android_resources"])
|
|
58
|
+
|
|
59
|
+
def test_long_silent_build_emits_only_safe_progress(self):
|
|
60
|
+
code, rows, _ = self.invoke("import time; time.sleep(.12)", interval=.03)
|
|
61
|
+
self.assertEqual(code, 0)
|
|
62
|
+
self.assertTrue(rows)
|
|
63
|
+
for row in rows:
|
|
64
|
+
self.assertEqual(row["schema"], "gowalk-cicd/android-build-progress.v1")
|
|
65
|
+
self.assertEqual(row["phase"], "flutter_bundle")
|
|
66
|
+
|
|
67
|
+
def test_missing_program_has_typed_failure_without_argv(self):
|
|
68
|
+
output = io.StringIO()
|
|
69
|
+
with contextlib.redirect_stdout(output):
|
|
70
|
+
code = diagnostics.run("gradle_bundle", ["/missing/private-program"])
|
|
71
|
+
self.assertEqual(code, 127)
|
|
72
|
+
self.assertNotIn("private-program", output.getvalue())
|
|
73
|
+
self.assertIn('"signals":["command_start_failed"]', output.getvalue())
|
|
74
|
+
|
|
75
|
+
def test_cli_forwards_exact_argument_vector(self):
|
|
76
|
+
with mock.patch.object(diagnostics, "run", return_value=7) as run:
|
|
77
|
+
self.assertEqual(diagnostics.main(["--phase", "gradle_bundle", "--", "./gradlew", "a b"]), 7)
|
|
78
|
+
run.assert_called_once_with("gradle_bundle", ["./gradlew", "a b"])
|
|
79
|
+
|
|
80
|
+
def test_real_release_step_keeps_proxy_and_compiler_arguments_on_failure(self):
|
|
81
|
+
root = Path(__file__).resolve().parents[1]
|
|
82
|
+
action = yaml.safe_load((root / "action.yml").read_text())
|
|
83
|
+
step = next(row for row in action["runs"]["steps"] if row["name"] == "Build release Android App Bundle")
|
|
84
|
+
with tempfile.TemporaryDirectory() as directory, upstream() as (proxy, _seen):
|
|
85
|
+
cwd = Path(directory)
|
|
86
|
+
flutter = cwd / "flutter"
|
|
87
|
+
flutter.write_text(f"#!{sys.executable}\nimport os,sys,json\nfrom pathlib import Path\n"
|
|
88
|
+
"assert os.environ['HTTPS_PROXY'].startswith('http://127.0.0.1:')\n"
|
|
89
|
+
"assert os.environ['NO_PROXY']==''\n"
|
|
90
|
+
"Path('arguments.json').write_text(json.dumps(sys.argv[1:]))\n"
|
|
91
|
+
"print('Missing classes detected while running R8: private-class')\nsys.exit(41)\n")
|
|
92
|
+
flutter.chmod(0o700)
|
|
93
|
+
env = {**os.environ, "GOOGLE_STORE_PROXY_URL": proxy, "PATH": directory + os.pathsep + os.environ["PATH"],
|
|
94
|
+
"ANDROID_BUILD_NUMBER": "17", "BUILD_NAME": "", "DART_DEFINES": "",
|
|
95
|
+
"DART_SYMBOLS_DIR": str(cwd / "symbols"), "NO_PROXY": "*", "HTTPS_PROXY": "http://wrong.invalid"}
|
|
96
|
+
script = step["run"].replace("${{ github.action_path }}", str(root))
|
|
97
|
+
result = subprocess.run(["bash", "-eo", "pipefail", "-c", script], cwd=cwd, env=env,
|
|
98
|
+
capture_output=True, text=True, timeout=20)
|
|
99
|
+
args = json.loads((cwd / "arguments.json").read_text())
|
|
100
|
+
self.assertEqual(result.returncode, 41, result.stdout + result.stderr)
|
|
101
|
+
self.assertIn("--obfuscate", args)
|
|
102
|
+
self.assertIn("--split-debug-info", args)
|
|
103
|
+
self.assertIn('"signals":["r8_missing_classes"]', result.stdout)
|
|
104
|
+
self.assertNotIn("private-class", result.stdout + result.stderr)
|
|
105
|
+
|
|
106
|
+
|
|
107
|
+
if __name__ == "__main__":
|
|
108
|
+
unittest.main()
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
"""A later job resumes retained public SDK bytes without bypassing digest or account checks."""
|
|
2
|
+
import hashlib
|
|
3
|
+
import os
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
import shutil
|
|
6
|
+
import subprocess
|
|
7
|
+
import sys
|
|
8
|
+
import tempfile
|
|
9
|
+
import unittest
|
|
10
|
+
from unittest.mock import patch
|
|
11
|
+
|
|
12
|
+
from flutter_manifest_fixture import tls_proxy
|
|
13
|
+
import test_flutter_archive as archive_fixture
|
|
14
|
+
|
|
15
|
+
ARCHIVE, ROOT = archive_fixture.ARCHIVE, archive_fixture.ROOT
|
|
16
|
+
DIGEST = hashlib.sha256(ARCHIVE).hexdigest()
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
def invoke(root, server, cert, **kwargs):
|
|
20
|
+
with patch.dict(os.environ, {"GOWALK_FLUTTER_ARCHIVE_CACHE": str(root / "gowalk-flutter-archives")}):
|
|
21
|
+
return archive_fixture.ArchiveRecoveryTests().invoke(root, server, cert, **kwargs)
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
class ArchiveCacheTests(unittest.TestCase):
|
|
25
|
+
def test_later_job_resumes_after_the_previous_job_exhausted_its_retries(self):
|
|
26
|
+
with tempfile.TemporaryDirectory() as first, tempfile.TemporaryDirectory() as second:
|
|
27
|
+
root, resumed = Path(first), Path(second)
|
|
28
|
+
with tls_proxy(root, ["partial", "range_partial", "range_partial", "range_partial"],
|
|
29
|
+
ARCHIVE) as (server, cert):
|
|
30
|
+
result = invoke(root, server, cert)
|
|
31
|
+
self.assertEqual((result.returncode, result.stdout), (18, b""))
|
|
32
|
+
partial = root / "gowalk-flutter-archives" / DIGEST / "download"
|
|
33
|
+
self.assertEqual(partial.read_bytes(), ARCHIVE[:124])
|
|
34
|
+
shutil.copytree(root / "gowalk-flutter-archives", resumed / "gowalk-flutter-archives")
|
|
35
|
+
with tls_proxy(resumed, ["range"], ARCHIVE) as (server, cert):
|
|
36
|
+
result = invoke(resumed, server, cert)
|
|
37
|
+
self.assertEqual((result.returncode, result.stdout), (0, ARCHIVE))
|
|
38
|
+
self.assertEqual(server.ranges, ["bytes=124-"])
|
|
39
|
+
self.assertFalse((resumed / "gowalk-flutter-archives" / DIGEST / "download").exists())
|
|
40
|
+
|
|
41
|
+
def test_complete_cached_archive_is_verified_without_another_archive_request(self):
|
|
42
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
43
|
+
root = Path(directory)
|
|
44
|
+
partial = root / "gowalk-flutter-archives" / DIGEST / "download"
|
|
45
|
+
partial.parent.mkdir(parents=True)
|
|
46
|
+
partial.write_bytes(ARCHIVE)
|
|
47
|
+
with tls_proxy(root, [], ARCHIVE) as (server, cert):
|
|
48
|
+
result = invoke(root, server, cert)
|
|
49
|
+
self.assertEqual((result.returncode, result.stdout, server.connects), (0, ARCHIVE, 0))
|
|
50
|
+
|
|
51
|
+
def test_complete_cached_bytes_cannot_bypass_a_missing_proxy(self):
|
|
52
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
53
|
+
root = Path(directory)
|
|
54
|
+
partial = root / "gowalk-flutter-archives" / DIGEST / "download"
|
|
55
|
+
partial.parent.mkdir(parents=True)
|
|
56
|
+
partial.write_bytes(ARCHIVE)
|
|
57
|
+
env = {**os.environ, "RUNNER_TEMP": str(root), "GOWALK_FLUTTER_ARCHIVE_SHA256": DIGEST,
|
|
58
|
+
"GOWALK_FLUTTER_ARCHIVE_CACHE": str(root / "gowalk-flutter-archives"),
|
|
59
|
+
"HTTPS_PROXY": "", "https_proxy": ""}
|
|
60
|
+
url = "https://provider.invalid/flutter_infra_release/releases/stable/sdk.tar.xz"
|
|
61
|
+
result = subprocess.run([sys.executable, str(ROOT / "flutter_manifest.py"),
|
|
62
|
+
"--connect-timeout", "15", "--retry", "5", url],
|
|
63
|
+
env=env, capture_output=True, timeout=10)
|
|
64
|
+
self.assertEqual((result.returncode, result.stdout), (1, b""))
|
|
65
|
+
self.assertEqual(partial.read_bytes(), ARCHIVE)
|
|
66
|
+
|
|
67
|
+
def test_changed_digest_cannot_reuse_another_release_partial(self):
|
|
68
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
69
|
+
root = Path(directory)
|
|
70
|
+
foreign = root / "gowalk-flutter-archives" / ("a" * 64) / "download"
|
|
71
|
+
foreign.parent.mkdir(parents=True)
|
|
72
|
+
foreign.write_bytes(b"unrelated release")
|
|
73
|
+
with tls_proxy(root, ["ok"], ARCHIVE) as (server, cert):
|
|
74
|
+
result = invoke(root, server, cert)
|
|
75
|
+
self.assertEqual((result.returncode, result.stdout), (0, ARCHIVE))
|
|
76
|
+
self.assertEqual(server.ranges, [""])
|
|
77
|
+
self.assertEqual(foreign.read_bytes(), b"unrelated release")
|
|
78
|
+
|
|
79
|
+
def test_corrupted_restored_partial_fails_verification_and_is_not_cached_again(self):
|
|
80
|
+
with tempfile.TemporaryDirectory() as directory:
|
|
81
|
+
root = Path(directory)
|
|
82
|
+
partial = root / "gowalk-flutter-archives" / DIGEST / "download"
|
|
83
|
+
partial.parent.mkdir(parents=True)
|
|
84
|
+
partial.write_bytes(b"wrong bytes")
|
|
85
|
+
with tls_proxy(root, ["range"], ARCHIVE) as (server, cert):
|
|
86
|
+
result = invoke(root, server, cert)
|
|
87
|
+
self.assertEqual((result.returncode, result.stdout), (1, b""))
|
|
88
|
+
self.assertEqual(server.ranges, ["bytes=11-"])
|
|
89
|
+
self.assertFalse(partial.exists())
|
|
90
|
+
|
|
91
|
+
def test_symlinked_cache_does_not_read_or_write_another_directory(self):
|
|
92
|
+
with tempfile.TemporaryDirectory() as directory, tempfile.TemporaryDirectory() as outside:
|
|
93
|
+
root = Path(directory)
|
|
94
|
+
(root / "gowalk-flutter-archives").symlink_to(outside, target_is_directory=True)
|
|
95
|
+
with tls_proxy(root, [], ARCHIVE) as (server, cert):
|
|
96
|
+
result = invoke(root, server, cert)
|
|
97
|
+
self.assertEqual((result.returncode, result.stdout, server.connects), (1, b"", 0))
|
|
98
|
+
self.assertEqual(list(Path(outside).iterdir()), [])
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
if __name__ == "__main__":
|
|
102
|
+
unittest.main()
|
|
@@ -24,7 +24,11 @@ class BootstrapWiringTests(unittest.TestCase):
|
|
|
24
24
|
self.assertNotIn("continue-on-error", step)
|
|
25
25
|
self.assertEqual(checked, 3)
|
|
26
26
|
setup = yaml.safe_load((ROOT / "android-action/flutter-setup/action.yml").read_text())
|
|
27
|
-
|
|
27
|
+
steps = setup["runs"]["steps"]
|
|
28
|
+
by_id = {step.get("id"): step for step in steps}
|
|
29
|
+
guard, scope, resolve, sdk_cache, pub_cache, install = (
|
|
30
|
+
by_id[name] for name in ("transport", "scope", "flutter-action", "cache-flutter", "cache-pub", "install"))
|
|
31
|
+
save = next(step for step in steps if step["name"] == "Save installed Flutter SDK")
|
|
28
32
|
self.assertIn('play_store_proxy.py" --bootstrap-env', guard["run"])
|
|
29
33
|
self.assertNotIn("continue-on-error", guard)
|
|
30
34
|
self.assertTrue(scope["continue-on-error"])
|
|
@@ -32,9 +36,13 @@ class BootstrapWiringTests(unittest.TestCase):
|
|
|
32
36
|
self.assertEqual(resolve["with"]["dry-run"], "true")
|
|
33
37
|
for step in (resolve, install):
|
|
34
38
|
self.assertEqual(step["uses"], "subosito/flutter-action@1a449444c387b1966244ae4d4f8c696479add0b2")
|
|
35
|
-
|
|
39
|
+
expected_env = {
|
|
36
40
|
"BASH_ENV": "${{ steps.transport.outputs.manifest_env }}",
|
|
37
|
-
"GOWALK_FLUTTER_MANIFEST_HELPER": "${{ steps.transport.outputs.manifest_helper }}"}
|
|
41
|
+
"GOWALK_FLUTTER_MANIFEST_HELPER": "${{ steps.transport.outputs.manifest_helper }}"}
|
|
42
|
+
if step is install:
|
|
43
|
+
expected_env["GOWALK_FLUTTER_ARCHIVE_CACHE"] = (
|
|
44
|
+
"${{ inputs.cache == 'true' && format('{0}/gowalk-flutter-archives', runner.temp) || '' }}")
|
|
45
|
+
self.assertEqual(step["env"], expected_env)
|
|
38
46
|
self.assertEqual(step["with"]["cache"], "false")
|
|
39
47
|
self.assertEqual(step["with"]["pub-cache"], "false")
|
|
40
48
|
self.assertNotIn("continue-on-error", step)
|
|
@@ -66,7 +74,8 @@ class BootstrapWiringTests(unittest.TestCase):
|
|
|
66
74
|
self.assertEqual(restore["uses"],
|
|
67
75
|
"actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25")
|
|
68
76
|
install = next(step for step in steps if step.get("id") == "install")
|
|
69
|
-
save = steps[
|
|
77
|
+
save = next(step for step in steps if step["name"] == "Save installed Flutter SDK")
|
|
78
|
+
self.assertGreater(steps.index(save), steps.index(install))
|
|
70
79
|
self.assertEqual(save["uses"], "actions/cache/save@caa296126883cff596d87d8935842f9db880ef25")
|
|
71
80
|
self.assertEqual(save["with"], restore["with"])
|
|
72
81
|
self.assertTrue(save["continue-on-error"])
|
|
@@ -1 +1 @@
|
|
|
1
|
-
1.0.
|
|
1
|
+
1.0.98
|