@trillboards/edge-sdk 0.2.225 → 0.2.226
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/deploy/windows/README.md +9 -4
- package/deploy/windows/ci/test-rollout-proofs.ps1 +300 -38
- package/deploy/windows/install-update.ps1 +5 -2
- package/deploy/windows/register-detector-task.ps1 +19 -1
- package/dist/AlertClips.d.ts +16 -1
- package/dist/AlertClips.d.ts.map +1 -1
- package/dist/AlertClips.js +35 -2
- package/dist/AlertClips.js.map +1 -1
- package/dist/CloudSensingManager.d.ts +33 -0
- package/dist/CloudSensingManager.d.ts.map +1 -1
- package/dist/CloudSensingManager.js +57 -0
- package/dist/CloudSensingManager.js.map +1 -1
- package/dist/DurableVlmCandidateSpool.d.ts +17 -0
- package/dist/DurableVlmCandidateSpool.d.ts.map +1 -1
- package/dist/DurableVlmCandidateSpool.js +46 -0
- package/dist/DurableVlmCandidateSpool.js.map +1 -1
- package/dist/EdgeAgent.js +1 -1
- package/dist/VideoSegmentUploader.d.ts +28 -2
- package/dist/VideoSegmentUploader.d.ts.map +1 -1
- package/dist/VideoSegmentUploader.js +53 -6
- package/dist/VideoSegmentUploader.js.map +1 -1
- package/dist/autoSense.d.ts +11 -1
- package/dist/autoSense.d.ts.map +1 -1
- package/dist/autoSense.js +33 -2
- package/dist/autoSense.js.map +1 -1
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +5 -1
- package/dist/cli.js.map +1 -1
- package/dist/connectorUpdate.d.ts +189 -38
- package/dist/connectorUpdate.d.ts.map +1 -1
- package/dist/connectorUpdate.js +333 -48
- package/dist/connectorUpdate.js.map +1 -1
- package/dist/connectorUpdateApply.d.ts +23 -1
- package/dist/connectorUpdateApply.d.ts.map +1 -1
- package/dist/connectorUpdateApply.js +72 -1
- package/dist/connectorUpdateApply.js.map +1 -1
- package/dist/connectorUpdateApplyMac.d.ts +4 -0
- package/dist/connectorUpdateApplyMac.d.ts.map +1 -1
- package/dist/connectorUpdateApplyMac.js +14 -0
- package/dist/connectorUpdateApplyMac.js.map +1 -1
- package/dist/diagnose.d.ts +1 -1
- package/dist/diagnose.js +1 -1
- package/dist/localSensing.d.ts +37 -2
- package/dist/localSensing.d.ts.map +1 -1
- package/dist/localSensing.js +65 -2
- package/dist/localSensing.js.map +1 -1
- package/dist/sensingRunMode.d.ts +12 -0
- package/dist/sensingRunMode.d.ts.map +1 -1
- package/dist/sensingRunMode.js.map +1 -1
- package/package.json +3 -3
- package/retained/WebsiteAdmissionCaptureClient.cjs +31 -2
- package/retained/WebsiteAdmissionState.cjs +1 -1
- package/retained/__tests__/website-admission.test.cjs +27 -0
package/deploy/windows/README.md
CHANGED
|
@@ -148,10 +148,15 @@ correct, and read by nothing.
|
|
|
148
148
|
`.zip` and a `.json` cannot carry Authenticode, and a hash arriving over the
|
|
149
149
|
channel it protects is a corruption check, not an authenticity check.
|
|
150
150
|
- **Apply** — `apply-update.ps1 -Commit` hands the swap to `install-update.ps1`
|
|
151
|
-
unchanged. The connector picks the moment: it waits
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
151
|
+
unchanged. The connector picks the moment (0.2.226+): it waits only for work
|
|
152
|
+
a restart could cost an alert — an alert onset or wait/recheck follow-up the
|
|
153
|
+
server has not acknowledged, alert video still owed, a follow-up due within
|
|
154
|
+
150 s — then holds new looks while the ones in the model finish, and hands
|
|
155
|
+
over. Routine looks and the backlog are durable and resume after the restart.
|
|
156
|
+
After 30 min without such a moment it applies anyway. Rule, deadline and
|
|
157
|
+
horizon are runtime_config `CONNECTOR_UPDATE_SAFE_POINT_*`, live within one
|
|
158
|
+
profile poll. The new version reports when it was staged, when it was handed
|
|
159
|
+
over and which branch took it (`updater.detail`, from `.auto-update\handover.json`).
|
|
155
160
|
- **Task shape** — re-registration produces the best shape the CALLER can make,
|
|
156
161
|
and a headless S4U/boot shape needs an elevated caller. The applier probes
|
|
157
162
|
first and REFUSES the update rather than silently returning a headless machine
|
|
@@ -22,6 +22,28 @@
|
|
|
22
22
|
connector's exit to the new one's health marker (written once its
|
|
23
23
|
cameras are started), on PROOF 1's swap and PROOF 2's rollback.
|
|
24
24
|
|
|
25
|
+
-Scenario alert-safe-point runs PROOF 4 instead, on a second runner:
|
|
26
|
+
|
|
27
|
+
PROOF 4 an update on a SATURATED box applies within minutes and an alert
|
|
28
|
+
in flight at update time is still delivered on time. The box runs
|
|
29
|
+
website-managed retained review the way the Texas store PC does
|
|
30
|
+
(rollout-proof-rig.mjs --website: three alert cameras - a lasso
|
|
31
|
+
with a wait/recheck route, an immediate person alert, the
|
|
32
|
+
unattended-customer rule - and -IdleCameras idle ones, a stand-in
|
|
33
|
+
detector answering two looks at a time, every review ingested by
|
|
34
|
+
trillboard-api's real parser). After warm-up the harness switches
|
|
35
|
+
the alert ON (a person in the lasso), publishes B the moment the
|
|
36
|
+
lasso's wait anchor reaches the API, and keeps the alert ON across
|
|
37
|
+
the hand-over. PASS: B was staged while the onset and the
|
|
38
|
+
follow-up were still owed; every onset and the follow-up reached
|
|
39
|
+
the API before the hand-over and within 10 s of capture (the
|
|
40
|
+
follow-up by anchor + wait + 10 s); the box then handed over at a
|
|
41
|
+
safe point (not the deadline) while the alert was still ON, so a
|
|
42
|
+
continuing alert does not hold an update; B healthy and reporting
|
|
43
|
+
within -PublishDeadlineMinutes. Also recorded: the restart gap
|
|
44
|
+
(old exit -> new health marker; last -> first review per alert
|
|
45
|
+
camera) and how busy the detector was while B waited.
|
|
46
|
+
|
|
25
47
|
The two published builds are the release's payload relabelled (B, C =
|
|
26
48
|
version + 1000, + 1001) with a catalog signed by a rig certificate whose
|
|
27
49
|
subject is the production publisher's CN and which only this runner trusts;
|
|
@@ -40,7 +62,13 @@ param(
|
|
|
40
62
|
[Parameter(Mandatory = $true)][string]$PersonFrame,
|
|
41
63
|
[ValidateRange(1, 60)][int]$PublishDeadlineMinutes = 15,
|
|
42
64
|
[ValidateRange(1, 60)][int]$RollbackDeadlineMinutes = 15,
|
|
43
|
-
[ValidateRange(1, 30)][int]$NoRetryWatchMinutes = 6
|
|
65
|
+
[ValidateRange(1, 30)][int]$NoRetryWatchMinutes = 6,
|
|
66
|
+
[ValidateSet('update-path', 'alert-safe-point')][string]$Scenario = 'update-path',
|
|
67
|
+
# PROOF 4: cameras nobody watches, beside the three alert cameras.
|
|
68
|
+
[ValidateRange(1, 12)][int]$IdleCameras = 6,
|
|
69
|
+
# PROOF 4: the lasso route's wait. Longer than staging takes here (20-40 s) and no longer
|
|
70
|
+
# than the connector's 150 s follow-up horizon, so the whole wait is critical alert work.
|
|
71
|
+
[ValidateRange(30, 150)][int]$WaitSeconds = 120
|
|
44
72
|
)
|
|
45
73
|
$ErrorActionPreference = 'Stop'
|
|
46
74
|
Set-StrictMode -Version Latest
|
|
@@ -53,6 +81,12 @@ foreach ($p in @($UpdateZip, $PersonFrame)) { if (-not (Test-Path -LiteralPath $
|
|
|
53
81
|
$Here = Split-Path -Parent $MyInvocation.MyCommand.Path # deploy\windows\ci
|
|
54
82
|
$PkgRoot = (Resolve-Path (Join-Path $Here '..\..\..')).Path # packages\edge-sdk
|
|
55
83
|
$RigScript = Join-Path $PkgRoot 'scripts\rollout-proof-rig.mjs'
|
|
84
|
+
$AlertScenario = $Scenario -eq 'alert-safe-point'
|
|
85
|
+
# PROOF 4 ingests every review through the server's own parser and selection, from this checkout.
|
|
86
|
+
$ApiRoot = Join-Path (Resolve-Path (Join-Path $PkgRoot '..\..\..')).Path 'trillboard-api'
|
|
87
|
+
if ($AlertScenario -and -not (Test-Path -LiteralPath (Join-Path $ApiRoot 'services\retainedCctvReviewProtocol.js'))) {
|
|
88
|
+
throw "PROOF 4 needs the checkout's trillboard-api (not found at $ApiRoot)"
|
|
89
|
+
}
|
|
56
90
|
$installDir = Join-Path $env:LOCALAPPDATA 'Trillboards\edge'
|
|
57
91
|
$updateDir = Join-Path $installDir '.auto-update'
|
|
58
92
|
$connectorLog = Join-Path $installDir 'logs\connector.log'
|
|
@@ -66,7 +100,7 @@ $root = [IO.Path]::GetFullPath((Join-Path $env:RUNNER_TEMP ('tb-rollout-' + [Gui
|
|
|
66
100
|
$rigOut = Join-Path $root 'rig'; $www = Join-Path $root 'www'; $trees = Join-Path $root 'trees'; $statePath = Join-Path $root 'state.json'
|
|
67
101
|
New-Item -ItemType Directory -Path $root, $rigOut, $www, $trees | Out-Null
|
|
68
102
|
$timeline = Join-Path $root 'timeline.log'
|
|
69
|
-
$receipt = [ordered]@{ atUtc = [DateTimeOffset]::UtcNow.ToString('o'); githubRunId = $env:GITHUB_RUN_ID; passed = $false; phase = 'start' }
|
|
103
|
+
$receipt = [ordered]@{ atUtc = [DateTimeOffset]::UtcNow.ToString('o'); githubRunId = $env:GITHUB_RUN_ID; scenario = $Scenario; passed = $false; phase = 'start' }
|
|
70
104
|
$owned = New-Object System.Collections.Generic.List[System.Diagnostics.Process]
|
|
71
105
|
$trustedThumbprints = New-Object System.Collections.Generic.List[string]
|
|
72
106
|
$hostsOriginal = $null
|
|
@@ -170,10 +204,16 @@ function Read-JsonFile([string]$Path) {
|
|
|
170
204
|
if (-not (Test-Path -LiteralPath $Path)) { return $null }
|
|
171
205
|
try { return (Get-Content -LiteralPath $Path -Raw | ConvertFrom-Json) } catch { return $null }
|
|
172
206
|
}
|
|
207
|
+
# A property that may be absent, without tripping StrictMode.
|
|
208
|
+
function Get-Prop($Object, [string]$Name) {
|
|
209
|
+
if ($null -ne $Object -and $Object.PSObject.Properties.Name -contains $Name) { return $Object.$Name }
|
|
210
|
+
return $null
|
|
211
|
+
}
|
|
173
212
|
function Get-StatusReports {
|
|
174
213
|
$log = Join-Path $rigOut 'api-requests.jsonl'
|
|
175
214
|
if (-not (Test-Path -LiteralPath $log)) { return @() }
|
|
176
|
-
|
|
215
|
+
# Filter as text first: a website-managed box writes a request line every few seconds.
|
|
216
|
+
@(Get-Content -LiteralPath $log -ErrorAction SilentlyContinue | Where-Object { $_ -like '*connector-status*' } | ForEach-Object {
|
|
177
217
|
$raw = $_; $row = $null; try { $row = $raw | ConvertFrom-Json } catch { }
|
|
178
218
|
if ($row -and $row.url -like '*connector-status*' -and $row.body) {
|
|
179
219
|
$body = $null; try { $body = $row.body | ConvertFrom-Json } catch { }
|
|
@@ -205,12 +245,43 @@ function Update-LogEvents {
|
|
|
205
245
|
function First-LogEvent([string]$Pattern, $After) {
|
|
206
246
|
@($script:logEvents | Where-Object { $_.Line -match $Pattern -and ($null -eq $After -or $_.At -ge $After) } | Select-Object -First 1)
|
|
207
247
|
}
|
|
208
|
-
|
|
209
|
-
|
|
248
|
+
$script:stateManifest = $null; $script:stateAdvert = $null; $script:stateScene = 'clear'
|
|
249
|
+
function Write-State($Manifest, $Advert, [string]$Scene = $script:stateScene) {
|
|
250
|
+
$script:stateManifest = $Manifest; $script:stateAdvert = $Advert; $script:stateScene = $Scene
|
|
251
|
+
$json = [ordered]@{ manifest = $Manifest; advert = $Advert; scene = $Scene } | ConvertTo-Json -Depth 8
|
|
210
252
|
$tmp = "$statePath.tmp"
|
|
211
253
|
[IO.File]::WriteAllText($tmp, $json, [Text.UTF8Encoding]::new($false))
|
|
212
254
|
Move-Item -LiteralPath $tmp -Destination $statePath -Force
|
|
213
255
|
}
|
|
256
|
+
# PROOF 4: what the stand-in detector sees (rollout-proof-rig.mjs --website): 'clear' or 'lasso'.
|
|
257
|
+
function Set-Scene([string]$Scene) {
|
|
258
|
+
Write-State $script:stateManifest $script:stateAdvert $Scene
|
|
259
|
+
Note ("scene: " + $Scene)
|
|
260
|
+
}
|
|
261
|
+
# The doorbell as production sends it (cameraPairingController.connectorReleaseBody), with the
|
|
262
|
+
# server-settable safe-point policy (runtime_config CONNECTOR_UPDATE_SAFE_POINT_*).
|
|
263
|
+
function New-Advert([string]$Version, [string]$PublishedAt) {
|
|
264
|
+
[ordered]@{ windows = [ordered]@{ version = $Version; publishedAt = $PublishedAt }; macos = $null; minReleaseAgeMs = 0
|
|
265
|
+
safePoint = [ordered]@{ rule = 'alert_work'; deadlineMs = 1800000; followUpHorizonMs = 150000 } }
|
|
266
|
+
}
|
|
267
|
+
function Get-UnixMs($At) { if ($null -eq $At) { return $null }; return ([DateTimeOffset]$At).ToUnixTimeMilliseconds() }
|
|
268
|
+
# PROOF 4: the reviews the stand-in API accepted, read incrementally (rig reviews.jsonl). A line
|
|
269
|
+
# still being written is left for the next read.
|
|
270
|
+
$script:reviewLines = 0
|
|
271
|
+
$script:refusedReviews = 0
|
|
272
|
+
$script:reviews = New-Object System.Collections.Generic.List[object]
|
|
273
|
+
function Update-RigReviews {
|
|
274
|
+
$file = Join-Path $rigOut 'reviews.jsonl'
|
|
275
|
+
if (-not (Test-Path -LiteralPath $file)) { return }
|
|
276
|
+
$lines = @(Get-Content -LiteralPath $file -ErrorAction SilentlyContinue)
|
|
277
|
+
for ($i = $script:reviewLines; $i -lt $lines.Count; $i++) {
|
|
278
|
+
$row = $null
|
|
279
|
+
try { $row = $lines[$i] | ConvertFrom-Json } catch { }
|
|
280
|
+
if ($null -eq $row) { if ($i -eq $lines.Count - 1) { break } else { continue } }
|
|
281
|
+
if ($row.PSObject.Properties.Name -contains 'refused') { $script:refusedReviews++ } else { $script:reviews.Add($row) }
|
|
282
|
+
$script:reviewLines = $i + 1
|
|
283
|
+
}
|
|
284
|
+
}
|
|
214
285
|
|
|
215
286
|
try {
|
|
216
287
|
$receipt.phase = 'certificates'
|
|
@@ -276,23 +347,36 @@ try {
|
|
|
276
347
|
}
|
|
277
348
|
|
|
278
349
|
$treeB = Join-Path $trees 'B'; Copy-Item -LiteralPath $treeA -Destination $treeB -Recurse; Set-TreeVersion $treeB $versionB
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
350
|
+
if (-not $AlertScenario) {
|
|
351
|
+
$treeC = Join-Path $trees 'C'; Copy-Item -LiteralPath $treeB -Destination $treeC -Recurse; Set-TreeVersion $treeC $versionC
|
|
352
|
+
$cliC = Join-Path $treeC 'lib\edge-sdk\dist\cli.js'
|
|
353
|
+
$cliText = [IO.File]::ReadAllText($cliC)
|
|
354
|
+
$breaker = "if (process.argv.slice(2).some((a) => a === 'auto-sense' || a === 'local-sense')) { console.error('[rig] deliberately broken build " + $versionC + " exits at startup'); process.exit(3); }`n"
|
|
355
|
+
if ($cliText.StartsWith('#!')) { $nl = $cliText.IndexOf("`n"); $cliText = $cliText.Substring(0, $nl + 1) + $breaker + $cliText.Substring($nl + 1) } else { $cliText = $breaker + $cliText }
|
|
356
|
+
[IO.File]::WriteAllText($cliC, $cliText, [Text.UTF8Encoding]::new($false))
|
|
357
|
+
}
|
|
285
358
|
$entryA = Publish-Tree $treeA $versionA
|
|
286
|
-
Note ("payloads: installed $versionA, published $versionB
|
|
359
|
+
if ($AlertScenario) { Note ("payloads: installed $versionA, published $versionB (catalogs signed by the rig's CN=Aethercorp Inc)") }
|
|
360
|
+
else { Note ("payloads: installed $versionA, published $versionB, broken $versionC (catalogs signed by the rig's CN=Aethercorp Inc)") }
|
|
287
361
|
|
|
288
362
|
$receipt.phase = 'rig'
|
|
289
363
|
$hostsOriginal = [IO.File]::ReadAllText($hostsPath)
|
|
290
364
|
[IO.File]::WriteAllText($hostsPath, $hostsOriginal.TrimEnd() + "`r`n127.0.0.1 screen.trillboards.com # tb-rollout-proof`r`n", [Text.Encoding]::ASCII)
|
|
291
365
|
& ipconfig.exe /flushdns | Out-Null
|
|
292
|
-
Write-State (New-Channel @($entryA)) (
|
|
366
|
+
Write-State (New-Channel @($entryA)) (New-Advert $versionA $entryA.publishedAt) 'clear'
|
|
293
367
|
$systemNode = (Get-Command node.exe -ErrorAction Stop).Source
|
|
294
|
-
$
|
|
295
|
-
|
|
368
|
+
$rigArgs = @($RigScript, '--out', $rigOut, '--state', $statePath, '--www', $www, '--api-port', '47393', '--key', $keyPem, '--cert', $certPem)
|
|
369
|
+
$websiteConfig = Join-Path $root 'website.config.json'
|
|
370
|
+
$cameraCount = 1
|
|
371
|
+
if ($AlertScenario) {
|
|
372
|
+
# The saturated website-managed store PC: the payload's own stub model, a stand-in detector,
|
|
373
|
+
# the server's own retained selection and review parser from this checkout.
|
|
374
|
+
$cameraCount = 3 + $IdleCameras
|
|
375
|
+
$rigArgs += @('--website', '--api-root', $ApiRoot, '--dist', (Join-Path $treeA 'lib\edge-sdk\dist'),
|
|
376
|
+
'--rtsp-base', 'rtsp://127.0.0.1:8554/', '--idle-cameras', [string]$IdleCameras, '--wait-seconds', [string]$WaitSeconds,
|
|
377
|
+
'--write-config', $websiteConfig)
|
|
378
|
+
}
|
|
379
|
+
$owned.Add((Start-Process -FilePath $systemNode -ArgumentList (($rigArgs | ForEach-Object { '"' + $_ + '"' }) -join ' ') -PassThru -WindowStyle Hidden `
|
|
296
380
|
-RedirectStandardOutput (Join-Path $root 'rig.log') -RedirectStandardError (Join-Path $root 'rig.err.log')))
|
|
297
381
|
$deadline = (Utc).AddSeconds(30)
|
|
298
382
|
while (-not (Test-Path -LiteralPath (Join-Path $rigOut 'rig-ready.json')) -and (Utc) -lt $deadline) { Start-Sleep -Milliseconds 300 }
|
|
@@ -309,6 +393,11 @@ try {
|
|
|
309
393
|
if ((Get-FileHash -LiteralPath $archive -Algorithm SHA256).Hash.ToLowerInvariant() -ne 'a84224f5a490855684e62ba6fb71e80ea2a5e16aed61883bb58380290e93e467') { throw 'MediaMTX checksum mismatch' }
|
|
310
394
|
Expand-Archive -LiteralPath $archive -DestinationPath $cam
|
|
311
395
|
$rtspPort = 8554; $mtxApiPort = 9997
|
|
396
|
+
# PROOF 4: every camera path re-serves one published moving pattern (one encode, N RTSP
|
|
397
|
+
# streams), as test-detector-looks-field.ps1 does, so the box always has a changed frame.
|
|
398
|
+
$paths = if ($AlertScenario) {
|
|
399
|
+
" src:`n" + ((1..$cameraCount | ForEach-Object { " cam$($_):`n source: rtsp://127.0.0.1:$rtspPort/src`n sourceOnDemand: no" }) -join "`n")
|
|
400
|
+
} else { ' cam:' }
|
|
312
401
|
@"
|
|
313
402
|
logLevel: warn
|
|
314
403
|
rtspAddress: 127.0.0.1:$rtspPort
|
|
@@ -331,7 +420,7 @@ authInternalUsers:
|
|
|
331
420
|
- action: read
|
|
332
421
|
- action: api
|
|
333
422
|
paths:
|
|
334
|
-
|
|
423
|
+
$paths
|
|
335
424
|
"@ | Set-Content -LiteralPath (Join-Path $cam 'mediamtx.yml') -Encoding utf8
|
|
336
425
|
$owned.Add((Start-Process -FilePath (Join-Path $cam 'mediamtx.exe') -ArgumentList ('"' + (Join-Path $cam 'mediamtx.yml') + '"') -WorkingDirectory $cam -PassThru -WindowStyle Hidden -RedirectStandardOutput (Join-Path $root 'mediamtx.log') -RedirectStandardError (Join-Path $root 'mediamtx.err.log')))
|
|
337
426
|
Start-Sleep -Seconds 2
|
|
@@ -339,28 +428,43 @@ paths:
|
|
|
339
428
|
# outlives every swap (stop-existing.ps1 stops everything under the root).
|
|
340
429
|
$cameraFfmpeg = Join-Path $cam 'camera-ffmpeg.exe'
|
|
341
430
|
Copy-Item -LiteralPath (Join-Path $treeA 'bin\ffmpeg.exe') -Destination $cameraFfmpeg
|
|
342
|
-
|
|
343
|
-
|
|
431
|
+
if ($AlertScenario) {
|
|
432
|
+
$publishArgs = '-hide_banner -loglevel error -re -f lavfi -i testsrc2=size=640x360:rate=15 -c:v libx264 -preset ultrafast -tune zerolatency -pix_fmt yuv420p -g 30 -threads 1 -f rtsp -rtsp_transport tcp rtsp://127.0.0.1:' + $rtspPort + '/src'
|
|
433
|
+
} else {
|
|
434
|
+
$frame = Join-Path $cam 'scene.jpg'; Copy-Item -LiteralPath $PersonFrame -Destination $frame
|
|
435
|
+
$publishArgs = '-hide_banner -loglevel error -re -loop 1 -framerate 4 -i "' + $frame + '" -an -c:v libx264 -preset ultrafast -tune zerolatency -pix_fmt yuv420p -r 4 -g 8 -threads 1 -f rtsp -rtsp_transport tcp rtsp://127.0.0.1:' + $rtspPort + '/cam'
|
|
436
|
+
}
|
|
344
437
|
$owned.Add((Start-Process -FilePath $cameraFfmpeg -ArgumentList $publishArgs -PassThru -WindowStyle Hidden -RedirectStandardOutput (Join-Path $root 'camera.log') -RedirectStandardError (Join-Path $root 'camera.err.log')))
|
|
345
|
-
$deadline = (Utc).AddSeconds(
|
|
438
|
+
$deadline = (Utc).AddSeconds(45); $ready = 0
|
|
346
439
|
do {
|
|
347
440
|
Start-Sleep -Seconds 1
|
|
348
|
-
try { $ready = @((Invoke-RestMethod -Uri ('http://127.0.0.1:' + $mtxApiPort + '/v3/paths/list') -TimeoutSec 3).items | Where-Object { $_.name -
|
|
349
|
-
} while (-
|
|
350
|
-
if (-
|
|
351
|
-
Note (
|
|
441
|
+
try { $ready = @((Invoke-RestMethod -Uri ('http://127.0.0.1:' + $mtxApiPort + '/v3/paths/list') -TimeoutSec 3).items | Where-Object { $_.name -like 'cam*' -and $_.ready }).Count } catch { }
|
|
442
|
+
} while ($ready -lt $cameraCount -and (Utc) -lt $deadline)
|
|
443
|
+
if ($ready -lt $cameraCount) { throw "Only $ready of $cameraCount RTSP camera(s) became readable" }
|
|
444
|
+
if ($AlertScenario) { Note ("$cameraCount cameras live at rtsp://127.0.0.1:$rtspPort/cam1..cam$cameraCount") }
|
|
445
|
+
else { Note ('camera live at rtsp://127.0.0.1:' + $rtspPort + '/cam') }
|
|
352
446
|
|
|
353
447
|
$receipt.phase = 'install'
|
|
354
448
|
New-Item -ItemType Directory -Force -Path $installDir | Out-Null
|
|
355
449
|
foreach ($part in @('bin', 'lib')) { Copy-Item -LiteralPath (Join-Path $treeA $part) -Destination (Join-Path $installDir $part) -Recurse }
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
450
|
+
if ($AlertScenario) {
|
|
451
|
+
# The rig's paired config: one token per camera, no cloudSensing block, so the connector
|
|
452
|
+
# takes the ordinary website-managed retained review path (configuredWebsiteReview).
|
|
453
|
+
if (-not (Test-Path -LiteralPath $websiteConfig)) { throw 'The website rig wrote no paired config (see rig.err.log)' }
|
|
454
|
+
Copy-Item -LiteralPath $websiteConfig -Destination (Join-Path $installDir 'trillboards.config.json')
|
|
455
|
+
$rigReady = Get-Content -LiteralPath (Join-Path $rigOut 'rig-ready.json') -Raw | ConvertFrom-Json
|
|
456
|
+
$lassoField = [string]$rigReady.lassoField
|
|
457
|
+
if ($lassoField -notmatch '^alert_focus_[a-p]{16}$') { throw "The website rig named no lasso field ($lassoField)" }
|
|
458
|
+
} else {
|
|
459
|
+
$tokens = 1..2 | ForEach-Object { -join ((1..32) | ForEach-Object { '{0:x2}' -f (Get-Random -Maximum 256) }) }
|
|
460
|
+
[ordered]@{
|
|
461
|
+
apiBaseUrl = 'http://127.0.0.1:47393'
|
|
462
|
+
deviceToken = $tokens[0]
|
|
463
|
+
cameras = @([ordered]@{ deviceToken = $tokens[1]; screenId = 'tb-e2e-rollout'; rtspUrl = ('rtsp://127.0.0.1:' + $rtspPort + '/cam'); sourceType = 'direct_url'; transport = 'tcp' })
|
|
464
|
+
localSensing = [ordered]@{ enabled = $true; litertLmBaseUrl = 'http://127.0.0.1:9379' }
|
|
465
|
+
cloudSensing = [ordered]@{ candidateBuffer = [ordered]@{ continuousReview = [ordered]@{ enabled = $false } } }
|
|
466
|
+
} | ConvertTo-Json -Depth 6 | Set-Content -LiteralPath (Join-Path $installDir 'trillboards.config.json') -Encoding utf8
|
|
467
|
+
}
|
|
364
468
|
$powershell = Join-Path $env:WINDIR 'System32\WindowsPowerShell\v1.0\powershell.exe'
|
|
365
469
|
$null = Invoke-Owned -File $powershell -RawArguments ('-NoProfile -NonInteractive -ExecutionPolicy Bypass -File "' + (Join-Path $installDir 'bin\register-task.ps1') + '" -InstallDir "' + $installDir + '" -Mode auto-sense') -LogName 'register-task.log' -TimeoutSeconds 300
|
|
366
470
|
$task = Get-ScheduledTask -TaskName 'TrillboardsEdge' -TaskPath '\' -ErrorAction Stop
|
|
@@ -368,7 +472,9 @@ paths:
|
|
|
368
472
|
$installStarted = Utc
|
|
369
473
|
Note ("TrillboardsEdge registered and started on $versionA")
|
|
370
474
|
|
|
371
|
-
|
|
475
|
+
# A website-managed box checks every camera's capability and profile before it starts.
|
|
476
|
+
$healthyMinutes = if ($AlertScenario) { 8 } else { 5 }
|
|
477
|
+
$deadline = $installStarted.AddMinutes($healthyMinutes); $aHealthy = $null
|
|
372
478
|
while ((Utc) -lt $deadline) {
|
|
373
479
|
Start-Sleep -Seconds 1; Update-LogEvents
|
|
374
480
|
$marker = Read-JsonFile (Join-Path $updateDir 'health.json')
|
|
@@ -377,17 +483,18 @@ paths:
|
|
|
377
483
|
$aHealthy = Get-MarkerTime (Join-Path $updateDir 'health.json'); break
|
|
378
484
|
}
|
|
379
485
|
}
|
|
380
|
-
if (-not $aHealthy) { throw "$versionA never reported healthy and its version within
|
|
486
|
+
if (-not $aHealthy) { throw "$versionA never reported healthy and its version within $healthyMinutes minutes of its task start" }
|
|
381
487
|
$receipt.installedHealthySeconds = Seconds $installStarted $aHealthy
|
|
382
488
|
Note ("$versionA healthy $($receipt.installedHealthySeconds) s after its task started, and reporting")
|
|
383
489
|
|
|
490
|
+
if (-not $AlertScenario) {
|
|
384
491
|
# -------------------------------------------------------------------------
|
|
385
492
|
# PROOF 1 + 3: a newly published build reaches the box.
|
|
386
493
|
# -------------------------------------------------------------------------
|
|
387
494
|
$receipt.phase = 'proof-1-publish'
|
|
388
495
|
$entryB = Publish-Tree $treeB $versionB
|
|
389
496
|
$published1 = Utc
|
|
390
|
-
Write-State (New-Channel @($entryB, $entryA)) (
|
|
497
|
+
Write-State (New-Channel @($entryB, $entryA)) (New-Advert $versionB $entryB.publishedAt)
|
|
391
498
|
Note ("PUBLISHED $versionB on the channel and the doorbell")
|
|
392
499
|
$before = @(Get-ConnectorProcesses); $oldExit1 = $null; $done1 = $null; $markerB = $null
|
|
393
500
|
$deadline = $published1.AddMinutes($PublishDeadlineMinutes + 2)
|
|
@@ -406,7 +513,7 @@ paths:
|
|
|
406
513
|
$healthyB = if ($markerTimeB) { $markerTimeB } else { $done1 }
|
|
407
514
|
$doorbell1 = First-LogEvent ([regex]::Escape("advertises connector $versionB")) $published1
|
|
408
515
|
$staging1 = First-LogEvent ([regex]::Escape("staging $versionB")) $published1
|
|
409
|
-
$applying1 = First-LogEvent 'connector is idle|applying anyway' $published1
|
|
516
|
+
$applying1 = First-LogEvent 'connector is idle|applying anyway|safe point after' $published1
|
|
410
517
|
$polls = @(Get-ProfilePolls | Where-Object { $_ -ge $published1 })
|
|
411
518
|
$receipt.proof1 = [ordered]@{
|
|
412
519
|
publishedToHealthyMinutes = [Math]::Round(($healthyB - $published1).TotalMinutes, 2)
|
|
@@ -432,7 +539,7 @@ paths:
|
|
|
432
539
|
$receipt.phase = 'proof-2-broken'
|
|
433
540
|
$entryC = Publish-Tree $treeC $versionC
|
|
434
541
|
$published2 = Utc
|
|
435
|
-
Write-State (New-Channel @($entryC, $entryB, $entryA)) (
|
|
542
|
+
Write-State (New-Channel @($entryC, $entryB, $entryA)) (New-Advert $versionC $entryC.publishedAt)
|
|
436
543
|
Note ("PUBLISHED broken $versionC")
|
|
437
544
|
$pidB = [int]$markerB.pid; $exitB = $null; $sawC = $false; $cPids = New-Object System.Collections.Generic.HashSet[int]; $rolledBack = $null; $markerBack = $null
|
|
438
545
|
$deadline = $published2.AddMinutes($RollbackDeadlineMinutes + 2)
|
|
@@ -467,7 +574,7 @@ paths:
|
|
|
467
574
|
}
|
|
468
575
|
if ($failedVersions -notcontains $versionC) { throw "PROOF 2 FAILED: failed-versions.json does not list $versionC" }
|
|
469
576
|
$healthyBack = if ($markerTimeBack) { $markerTimeBack } else { $rolledBack }
|
|
470
|
-
$applying2 = First-LogEvent 'connector is idle|applying anyway' $published2
|
|
577
|
+
$applying2 = First-LogEvent 'connector is idle|applying anyway|safe point after' $published2
|
|
471
578
|
Note ("rolled back to $versionB; outcome: $($outcome.detail)")
|
|
472
579
|
|
|
473
580
|
$receipt.phase = 'proof-2-no-retry'
|
|
@@ -500,6 +607,160 @@ paths:
|
|
|
500
607
|
$receipt.proof3.rollbackGapSeconds = Seconds $exitB $healthyBack
|
|
501
608
|
Note ("PROOF 2: $versionC rolled back to $versionB $($receipt.proof2.handoffToRolledBackSeconds) s after the hand-off; not retried in $NoRetryWatchMinutes min ($pollsDuringWatch profile polls); reported: $($receipt.proof2.heldBackReport)")
|
|
502
609
|
Note ("PROOF 3: monitoring gap during the failed update and rollback: $($receipt.proof3.rollbackGapSeconds) s")
|
|
610
|
+
} else {
|
|
611
|
+
# -------------------------------------------------------------------------
|
|
612
|
+
# PROOF 4: a saturated box takes a release at a safe point without delaying
|
|
613
|
+
# an alert that is in flight when the release arrives.
|
|
614
|
+
# -------------------------------------------------------------------------
|
|
615
|
+
$receipt.phase = 'proof-4-warmup'
|
|
616
|
+
# B's payload is built now and served by the rig's CDN, but is not on the channel or the
|
|
617
|
+
# doorbell until the alert is in flight: publication itself is one state write.
|
|
618
|
+
$entryB = Publish-Tree $treeB $versionB
|
|
619
|
+
$alertKinds = @('lasso', 'person', 'paint')
|
|
620
|
+
$counts = @{}
|
|
621
|
+
$deadline = (Utc).AddMinutes(10); $warm = $false
|
|
622
|
+
while ((Utc) -lt $deadline) {
|
|
623
|
+
Start-Sleep -Seconds 2; Update-LogEvents; Update-RigReviews
|
|
624
|
+
foreach ($k in $alertKinds) { $counts[$k] = @($script:reviews | Where-Object { $_.kind -eq $k }).Count }
|
|
625
|
+
if (@($alertKinds | Where-Object { $counts[$_] -lt 5 }).Count -eq 0) { $warm = $true; break }
|
|
626
|
+
}
|
|
627
|
+
$countText = ($alertKinds | ForEach-Object { "$_ $($counts[$_])" }) -join ', '
|
|
628
|
+
if (-not $warm) { throw "PROOF 4 FAILED: the alert cameras were not looked at within 10 min of start (reviews: $countText; refused $($script:refusedReviews))" }
|
|
629
|
+
if ($script:refusedReviews -gt 0) { throw "PROOF 4 FAILED: the server's review parser refused $($script:refusedReviews) review(s) (rig reviews.jsonl)" }
|
|
630
|
+
Note ("warm: every alert camera is being looked at ($countText reviews)")
|
|
631
|
+
|
|
632
|
+
$receipt.phase = 'proof-4-alert'
|
|
633
|
+
Set-Scene 'lasso'; $alertOn = Utc; $alertOnMs = Get-UnixMs $alertOn
|
|
634
|
+
$anchor = $null; $deadline = $alertOn.AddMinutes(2)
|
|
635
|
+
while ((Utc) -lt $deadline -and -not $anchor) {
|
|
636
|
+
Start-Sleep -Milliseconds 500; Update-LogEvents; Update-RigReviews
|
|
637
|
+
$anchor = $script:reviews | Where-Object { $_.kind -eq 'lasso' -and $_.atMs -ge $alertOnMs -and (Get-Prop $_.fields $lassoField) -eq $true } | Select-Object -First 1
|
|
638
|
+
}
|
|
639
|
+
if (-not $anchor) { throw 'PROOF 4 FAILED: the alert opened no wait anchor on the lasso camera within 2 min' }
|
|
640
|
+
$followUpDueMs = [int64]$anchor.capturedAtMs + $WaitSeconds * 1000
|
|
641
|
+
Note ("ALERT: the lasso's wait anchor reached the API $([Math]::Round(([int64]$anchor.atMs - [int64]$anchor.capturedAtMs) / 1000, 1)) s after capture; its follow-up is due in $([Math]::Round(($followUpDueMs - (Get-UnixMs (Utc))) / 1000, 1)) s")
|
|
642
|
+
$published = Utc
|
|
643
|
+
$entryB['publishedAt'] = $published.ToString('yyyy-MM-ddTHH:mm:ss.fffffffZ')
|
|
644
|
+
Write-State (New-Channel @($entryB, $entryA)) (New-Advert $versionB $entryB.publishedAt) 'lasso'
|
|
645
|
+
Note ("PUBLISHED $versionB on the channel and the doorbell, with the alert in flight")
|
|
646
|
+
|
|
647
|
+
$receipt.phase = 'proof-4-handover'
|
|
648
|
+
$before = @(Get-ConnectorProcesses); $oldExit = $null; $clearAt = $null; $handoverRaw = $null; $done = $null; $markerTimeB = $null
|
|
649
|
+
$followUp = $null; $heldPastFollowUp = $false
|
|
650
|
+
$deadline = $published.AddMinutes($PublishDeadlineMinutes + 2)
|
|
651
|
+
while ((Utc) -lt $deadline) {
|
|
652
|
+
Start-Sleep -Milliseconds 500; Update-LogEvents; Update-RigReviews
|
|
653
|
+
# The old process writes the hand-over record as it hands over; the new one may clear it
|
|
654
|
+
# once it has reported it, so it is read the moment it appears.
|
|
655
|
+
if (-not $handoverRaw) {
|
|
656
|
+
$handoverPath = Join-Path $updateDir 'handover.json'
|
|
657
|
+
if (Test-Path -LiteralPath $handoverPath) {
|
|
658
|
+
try { $raw = [IO.File]::ReadAllText($handoverPath); if ($raw -match [regex]::Escape('"' + $versionB + '"')) { $handoverRaw = $raw; Note ('handover.json: ' + $raw) } } catch { }
|
|
659
|
+
}
|
|
660
|
+
}
|
|
661
|
+
if (-not $followUp) {
|
|
662
|
+
$followUp = $script:reviews | Where-Object { $_.kind -eq 'lasso' -and $_.slot -eq 'follow_up' -and [int64]$_.capturedAtMs -ge $followUpDueMs } | Select-Object -First 1
|
|
663
|
+
if ($followUp) { Note ("the lasso's follow-up reached the API $([Math]::Round(([int64]$followUp.atMs - $followUpDueMs) / 1000, 1)) s after it was due") }
|
|
664
|
+
}
|
|
665
|
+
$now = @(Get-ConnectorProcesses)
|
|
666
|
+
if (-not $oldExit -and $before.Count -gt 0 -and @($before | Where-Object { $now -contains $_ }).Count -eq 0) {
|
|
667
|
+
$oldExit = Utc; Note "old connector ($versionA) exited"
|
|
668
|
+
# The alert stayed ON across the hand-over. The new version starts on a clear scene.
|
|
669
|
+
if ($script:stateScene -ne 'clear') { Set-Scene 'clear'; $clearAt = Utc }
|
|
670
|
+
}
|
|
671
|
+
# A box that still holds the release 3 min after the follow-up is in is being held by the
|
|
672
|
+
# continuing alert. Switch it off so the run ends with a measurement, and fail on it below.
|
|
673
|
+
if (-not $oldExit -and $followUp -and $script:stateScene -ne 'clear' -and (Get-UnixMs (Utc)) -gt [int64]$followUp.atMs + 180000) {
|
|
674
|
+
$heldPastFollowUp = $true
|
|
675
|
+
Note 'no hand-over within 3 min of the follow-up while the alert continued: switching the alert off'
|
|
676
|
+
Set-Scene 'clear'; $clearAt = Utc
|
|
677
|
+
}
|
|
678
|
+
$marker = Read-JsonFile (Join-Path $updateDir 'health.json')
|
|
679
|
+
if ($marker -and [string]$marker.version -eq $versionB -and (Get-InstalledVersion) -eq $versionB -and
|
|
680
|
+
[int]$marker.pid -gt 0 -and (Get-Process -Id ([int]$marker.pid) -ErrorAction SilentlyContinue) -and
|
|
681
|
+
@(Get-StatusReports | Where-Object { [string]$_.Body.connectorVersion -eq $versionB }).Count -gt 0) {
|
|
682
|
+
$done = Utc; $markerTimeB = Get-MarkerTime (Join-Path $updateDir 'health.json'); break
|
|
683
|
+
}
|
|
684
|
+
}
|
|
685
|
+
if ($script:stateScene -ne 'clear') { Set-Scene 'clear'; if (-not $clearAt) { $clearAt = Utc } }
|
|
686
|
+
if (-not $done) { throw "PROOF 4 FAILED: $versionB was not installed, healthy and reporting within $($PublishDeadlineMinutes + 2) min of publication (installed: $(Get-InstalledVersion))" }
|
|
687
|
+
$healthyB = if ($markerTimeB) { $markerTimeB } else { $done }
|
|
688
|
+
|
|
689
|
+
# How the new version says it took the release (updater.detail on its first report).
|
|
690
|
+
$tookReport = $null; $detailDeadline = (Utc).AddSeconds(90)
|
|
691
|
+
do {
|
|
692
|
+
Update-LogEvents
|
|
693
|
+
$tookReport = Get-StatusReports | Where-Object { $_.At -ge $published -and [string]$_.Body.connectorVersion -eq $versionB -and
|
|
694
|
+
$null -ne (Get-Prop $_.Body 'updater') -and [string](Get-Prop $_.Body.updater 'state') -eq 'current' -and
|
|
695
|
+
[string](Get-Prop $_.Body.updater 'detail') -match ('^took ' + [regex]::Escape($versionB)) } | Select-Object -First 1
|
|
696
|
+
if ($tookReport) { break }
|
|
697
|
+
Start-Sleep -Seconds 2
|
|
698
|
+
} while ((Utc) -lt $detailDeadline)
|
|
699
|
+
|
|
700
|
+
# The instants, from the connector's own hand-over record where it has one.
|
|
701
|
+
$stagedLog = First-LogEvent ([regex]::Escape("$versionB is staged and verified")) $published
|
|
702
|
+
$waitLog = First-LogEvent ([regex]::Escape("$versionB waits for alert work")) $published
|
|
703
|
+
$drainLog = First-LogEvent ([regex]::Escape('no alert work')) $published
|
|
704
|
+
$safeLog = First-LogEvent ([regex]::Escape('safe point after')) $published
|
|
705
|
+
$deadlineLog = First-LogEvent ([regex]::Escape('applying anyway')) $published
|
|
706
|
+
$stagedAtMs = $null; $handedOverMs = $null; $branch = $null; $handoverSource = $null
|
|
707
|
+
if ($handoverRaw) {
|
|
708
|
+
$stagedAtMs = Get-UnixMs (Get-JsonTime $handoverRaw 'stagedAt')
|
|
709
|
+
$handedOverMs = Get-UnixMs (Get-JsonTime $handoverRaw 'handedOverAt')
|
|
710
|
+
$m = [regex]::Match($handoverRaw, '"branch"\s*:\s*"([a-z_]+)"'); if ($m.Success) { $branch = $m.Groups[1].Value }
|
|
711
|
+
if ($handedOverMs) { $handoverSource = 'handover.json' }
|
|
712
|
+
}
|
|
713
|
+
if (-not $stagedAtMs -and $stagedLog) { $stagedAtMs = Get-UnixMs $stagedLog[0].At }
|
|
714
|
+
if (-not $handedOverMs) {
|
|
715
|
+
if ($safeLog) { $handedOverMs = Get-UnixMs $safeLog[0].At; $handoverSource = 'log (safe point after)' }
|
|
716
|
+
elseif ($deadlineLog) { $handedOverMs = Get-UnixMs $deadlineLog[0].At; $handoverSource = 'log (applying anyway)' }
|
|
717
|
+
elseif ($oldExit) { $handedOverMs = Get-UnixMs $oldExit; $handoverSource = 'old process exit' }
|
|
718
|
+
}
|
|
719
|
+
if (-not $branch) { $branch = if ($safeLog) { 'safe_point' } elseif ($deadlineLog) { 'deadline' } else { $null } }
|
|
720
|
+
# What the old rule (nothing in flight AND nothing buffered) would still have waited for.
|
|
721
|
+
$notWaitedFor = $null
|
|
722
|
+
if ($safeLog) {
|
|
723
|
+
$m = [regex]::Match($safeLog[0].Line, 'not waited for: (\d+) look\(s\) in flight, (\d+) frame\(s\) held')
|
|
724
|
+
if ($m.Success) { $notWaitedFor = [ordered]@{ looksInFlight = [int]$m.Groups[1].Value; framesHeld = [int]$m.Groups[2].Value } }
|
|
725
|
+
}
|
|
726
|
+
$times = [ordered]@{
|
|
727
|
+
publishedAtMs = Get-UnixMs $published; stagedAtMs = $stagedAtMs; alertOnAtMs = $alertOnMs; clearAtMs = Get-UnixMs $clearAt
|
|
728
|
+
handoverAtMs = $handedOverMs; handoverSource = $handoverSource; branch = $branch; oldExitAtMs = Get-UnixMs $oldExit
|
|
729
|
+
healthyAtMs = Get-UnixMs $healthyB; reportedAtMs = Get-UnixMs $done; waitingLogged = [bool]$waitLog; waitSeconds = $WaitSeconds
|
|
730
|
+
}
|
|
731
|
+
$timesPath = Join-Path $root 'proof4-times.json'
|
|
732
|
+
$times | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $timesPath -Encoding utf8
|
|
733
|
+
$verdictPath = Join-Path $rigOut 'alert-verdict.json'
|
|
734
|
+
try {
|
|
735
|
+
$null = Invoke-Owned -File $systemNode -RawArguments ('"' + $RigScript + '" --alert-verdict "' + $timesPath + '" --out "' + $rigOut + '" --api-root "' + $ApiRoot + '"') -LogName 'proof4-verdict.log' -TimeoutSeconds 120
|
|
736
|
+
} catch { Note ('verdict: ' + $_.Exception.Message) }
|
|
737
|
+
if (-not (Test-Path -LiteralPath $verdictPath)) { throw 'PROOF 4 FAILED: the verdict was not written (see proof4-verdict.log)' }
|
|
738
|
+
$verdict = Get-Content -LiteralPath $verdictPath -Raw | ConvertFrom-Json
|
|
739
|
+
$receipt.proof4 = [ordered]@{
|
|
740
|
+
waitSeconds = $WaitSeconds
|
|
741
|
+
cameras = [ordered]@{ alert = $alertKinds; idle = $IdleCameras }
|
|
742
|
+
publishedToStagedSeconds = if ($stagedAtMs) { [Math]::Round(($stagedAtMs - (Get-UnixMs $published)) / 1000, 1) } else { $null }
|
|
743
|
+
stagedToHandoverSeconds = if ($stagedAtMs -and $handedOverMs) { [Math]::Round(($handedOverMs - $stagedAtMs) / 1000, 1) } else { $null }
|
|
744
|
+
publishedToHealthyMinutes = [Math]::Round(($healthyB - $published).TotalMinutes, 2)
|
|
745
|
+
handoverBranch = $branch
|
|
746
|
+
handoverSource = $handoverSource
|
|
747
|
+
alertOnAcrossHandover = (-not $heldPastFollowUp) -and $null -ne $clearAt -and $null -ne $oldExit -and $clearAt -ge $oldExit
|
|
748
|
+
waitLogged = if ($waitLog) { $waitLog[0].Line } else { $null }
|
|
749
|
+
drainLogged = if ($drainLog) { $drainLog[0].Line } else { $null }
|
|
750
|
+
safePointLogged = if ($safeLog) { $safeLog[0].Line } else { $null }
|
|
751
|
+
notWaitedFor = $notWaitedFor
|
|
752
|
+
restartGapSeconds = Seconds $oldExit $healthyB
|
|
753
|
+
newVersionReport = if ($tookReport) { [string]$tookReport.Body.updater.detail } else { $null }
|
|
754
|
+
verdict = $verdict
|
|
755
|
+
}
|
|
756
|
+
Note ("PROOF 4: $versionB staged $($receipt.proof4.publishedToStagedSeconds) s after publication with the alert in flight, handed over $($receipt.proof4.stagedToHandoverSeconds) s later at a $branch; healthy $($receipt.proof4.publishedToHealthyMinutes) min after publication; restart gap $($receipt.proof4.restartGapSeconds) s")
|
|
757
|
+
if ($notWaitedFor) { Note ("PROOF 4: the old rule would still have waited for $($notWaitedFor.looksInFlight) look(s) in flight and $($notWaitedFor.framesHeld) frame(s) held") }
|
|
758
|
+
if ($verdict.PSObject.Properties.Name -contains 'saturation' -and $verdict.saturation) { Note ("PROOF 4: while $versionB waited the detector was busy $($verdict.saturation.busyShare) of the time, $($verdict.saturation.meanConcurrent) look(s) in flight on average") }
|
|
759
|
+
if ($heldPastFollowUp) { throw "PROOF 4 FAILED: the continuing alert held $versionB for more than 3 min after the follow-up was in" }
|
|
760
|
+
if (-not $verdict.passed) { throw ('PROOF 4 FAILED: ' + (@($verdict.failures) -join '; ')) }
|
|
761
|
+
if (-not $tookReport) { throw "PROOF 4 FAILED: $versionB never reported how it took the release (updater.detail 'took $versionB ...')" }
|
|
762
|
+
if ([string]$tookReport.Body.updater.detail -notmatch 'at a safe point') { throw "PROOF 4 FAILED: $versionB reports it took the release otherwise: $($tookReport.Body.updater.detail)" }
|
|
763
|
+
}
|
|
503
764
|
$receipt.phase = 'complete'; $receipt.passed = $true
|
|
504
765
|
} catch {
|
|
505
766
|
$receipt.failure = $_.Exception.Message
|
|
@@ -531,7 +792,8 @@ paths:
|
|
|
531
792
|
$receipt | ConvertTo-Json -Depth 8 | Set-Content -LiteralPath (Join-Path $root 'receipt.json')
|
|
532
793
|
if ($env:GITHUB_OUTPUT) { ('artifact_path=' + $root) | Out-File -LiteralPath $env:GITHUB_OUTPUT -Append -Encoding utf8 }
|
|
533
794
|
if ($env:GITHUB_STEP_SUMMARY) {
|
|
534
|
-
|
|
795
|
+
$heading = if ($AlertScenario) { '### Connector update under an alert (rollout proof 4: saturated box, alert in flight)' } else { '### Connector update path (rollout proofs)' }
|
|
796
|
+
@($heading, '', '```json', ($receipt | ConvertTo-Json -Depth 8), '```') |
|
|
535
797
|
Out-File -LiteralPath $env:GITHUB_STEP_SUMMARY -Append -Encoding utf8
|
|
536
798
|
}
|
|
537
799
|
$receipt | ConvertTo-Json -Depth 8
|
|
@@ -374,8 +374,11 @@ function Update-DetectorTask([string]$Root) {
|
|
|
374
374
|
# elsewhere are a different boundary and must not be changed with this.
|
|
375
375
|
$registerArgs = @{ DetectorPath = $exe }
|
|
376
376
|
if ($budget) { $registerArgs.RamBudgetMb = $budget }
|
|
377
|
-
|
|
378
|
-
|
|
377
|
+
# -KeepRunningIfUnchanged: a detector whose task already runs this exact
|
|
378
|
+
# action keeps running. Stopping it here, after the new connector started,
|
|
379
|
+
# was a second minute of unanswered looks on every update.
|
|
380
|
+
& $register @registerArgs -KeepRunningIfUnchanged | Out-Null
|
|
381
|
+
Write-Host "Detector task checked$(if ($budget) { " (RAM ceiling $budget MiB)" })."
|
|
379
382
|
} catch {
|
|
380
383
|
Write-Warning "Could not migrate the detector task: $($_.Exception.Message). Detector-backed profiles need the local detector repaired; camera pairing is preserved."
|
|
381
384
|
}
|
|
@@ -45,7 +45,16 @@ param(
|
|
|
45
45
|
# for this task to say when the box could not be measured. A default here
|
|
46
46
|
# would make this file a second allocation authority - and one that cannot see
|
|
47
47
|
# how much memory the machine has.
|
|
48
|
-
[string]$RamBudgetMb = ""
|
|
48
|
+
[string]$RamBudgetMb = "",
|
|
49
|
+
# Leave a RUNNING detector alone when its task already runs exactly this
|
|
50
|
+
# action (same tool, port, log and RAM ceiling). install-update.ps1 passes it:
|
|
51
|
+
# it re-registers on every connector update to carry the ceiling, and the stop
|
|
52
|
+
# below cost every update a second minute of unanswered looks AFTER the new
|
|
53
|
+
# connector had started (Texas, 0.2.225 at 22:01Z on 2026-11-09: looks stopped
|
|
54
|
+
# 58 s for the connector restart, came back for one review, then stopped 62 s
|
|
55
|
+
# more while the detector restarted). A repair or a changed action still
|
|
56
|
+
# re-registers.
|
|
57
|
+
[switch]$KeepRunningIfUnchanged
|
|
49
58
|
)
|
|
50
59
|
$ErrorActionPreference = "Stop"
|
|
51
60
|
|
|
@@ -158,6 +167,15 @@ $Attempts = @(
|
|
|
158
167
|
# Disable the schedule and stop the verified native child before replacing the
|
|
159
168
|
# action. A hidden waiting host alone is not the service's lifetime boundary.
|
|
160
169
|
$priorService = & (Join-Path $PSScriptRoot 'stop-local-ai.ps1') -Service Detector -ToolPath $DetectorPath -Port ([int]$Port) -Inspect
|
|
170
|
+
if ($KeepRunningIfUnchanged -and $priorService.TaskPresent -and $priorService.Enabled -and $priorService.Running) {
|
|
171
|
+
$currentAction = @((Get-ScheduledTask -TaskName $TaskName -TaskPath '\' -ErrorAction SilentlyContinue).Actions) | Select-Object -First 1
|
|
172
|
+
if ($currentAction -and [string]$currentAction.Execute -and
|
|
173
|
+
[IO.Path]::GetFullPath([string]$currentAction.Execute) -ieq [IO.Path]::GetFullPath($wscript) -and
|
|
174
|
+
[string]$currentAction.Arguments -ieq $hiddenArgument) {
|
|
175
|
+
Write-Host "Detector task '$TaskName' already runs this action (tool, port, log, RAM ceiling); the running service is left alone."
|
|
176
|
+
return
|
|
177
|
+
}
|
|
178
|
+
}
|
|
161
179
|
$priorXml = if ($priorService.TaskPresent) { Export-ScheduledTask -TaskName $TaskName -TaskPath '\' } else { $null }
|
|
162
180
|
& (Join-Path $PSScriptRoot 'stop-local-ai.ps1') -Service Detector -ToolPath $DetectorPath -Port ([int]$Port) | Out-Null
|
|
163
181
|
try {
|
package/dist/AlertClips.d.ts
CHANGED
|
@@ -126,6 +126,17 @@ export declare class AlertClipReceiptPoller {
|
|
|
126
126
|
private readonly now;
|
|
127
127
|
private readonly log;
|
|
128
128
|
private target;
|
|
129
|
+
/**
|
|
130
|
+
* When the newest review that could have fired a new alert (an onset or a
|
|
131
|
+
* wait/recheck follow-up) was watched, until a receipt polled AFTER it comes
|
|
132
|
+
* back ingested (every receipt lists the camera's pending clip requests, so
|
|
133
|
+
* that one carries the alert's request if it fired), or RECEIPT_POLL_FOR_MS
|
|
134
|
+
* passes. Later non-critical watches neither extend nor clear it: on a camera
|
|
135
|
+
* whose condition persists every look is watched, and tying this to the
|
|
136
|
+
* current target kept it set for as long as the condition lasted. Read by
|
|
137
|
+
* connectorUpdate's `alert_work` safe point.
|
|
138
|
+
*/
|
|
139
|
+
private criticalSinceMs;
|
|
129
140
|
private timer;
|
|
130
141
|
/** When the scheduled poll fires (null while none is scheduled). */
|
|
131
142
|
private nextPollAtMs;
|
|
@@ -139,8 +150,12 @@ export declare class AlertClipReceiptPoller {
|
|
|
139
150
|
failures: number;
|
|
140
151
|
};
|
|
141
152
|
constructor(options: AlertClipReceiptPollerOptions);
|
|
153
|
+
/** Is a receipt that may carry a new alert's clip request still owed? */
|
|
154
|
+
hasCriticalWatch(): boolean;
|
|
142
155
|
/** Follow this review's receipt until the server has ingested it. */
|
|
143
|
-
watch(evidenceId: string
|
|
156
|
+
watch(evidenceId: string, opts?: {
|
|
157
|
+
critical?: boolean;
|
|
158
|
+
}): void;
|
|
144
159
|
stop(): void;
|
|
145
160
|
private schedule;
|
|
146
161
|
private poll;
|
package/dist/AlertClips.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"AlertClips.d.ts","sourceRoot":"","sources":["../src/AlertClips.ts"],"names":[],"mappings":"AAmCA,OAAO,EAAqB,KAAK,UAAU,EAAE,MAAM,yBAAyB,CAAC;AAC7E,OAAO,EAA0B,KAAK,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAEvF,+EAA+E;AAC/E,MAAM,WAAW,WAAW;IAC1B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,cAAc,EAAE,MAAM,CAAC;IACvB,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;CACtB;AAYD;;;GAGG;AACH,wBAAgB,gBAAgB,CAAC,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,WAAW,EAAE,CAmC/F;AAED;;;;;GAKG;AACH,wBAAgB,eAAe,CAAC,QAAQ,EAAE,WAAW,EAAE,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,WAAW,EAAE,CAYrH;AAOD,2DAA2D;AAC3D,wBAAgB,YAAY,CAAC,CAAC,EAAE;IAAE,QAAQ,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,MAAM,CAAC;IAAC,WAAW,EAAE,MAAM,CAAC;IAAC,GAAG,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,OAAO,CAAA;CAAE,GAAG,MAAM,EAAE,CAclI;AAED,MAAM,WAAW,OAAO;IACtB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,cAAc,EAAE,MAAM,CAAC;IACvB,YAAY,EAAE,MAAM,CAAC;IACrB,2CAA2C;IAC3C,cAAc,EAAE,MAAM,CAAC;CACxB;AAED,MAAM,MAAM,SAAS,GAAG;IAAE,IAAI,EAAE,OAAO,CAAA;CAAE,GAAG;IAAE,WAAW,EAAE,mBAAmB,GAAG,WAAW,GAAG,YAAY,CAAC;IAAC,MAAM,CAAC,EAAE,MAAM,CAAA;CAAE,CAAC;AAE/H,MAAM,MAAM,UAAU,GAAG,CACvB,IAAI,EAAE,MAAM,EACZ,IAAI,EAAE,MAAM,EAAE,EACd,OAAO,EAAE;IAAE,OAAO,EAAE,MAAM,CAAC;IAAC,WAAW,EAAE,OAAO,CAAA;CAAE,EAClD,QAAQ,EAAE,CAAC,KAAK,EAAE,KAAK,GAAG,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,KAAK,IAAI,KACpE,OAAO,CAAC;AAEb,+FAA+F;AAC/F,wBAAsB,cAAc,CAAC,CAAC,EAAE;IACtC,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,MAAM,EAAE,MAAM,CAAC;IACf,OAAO,EAAE,gBAAgB,CAAC;IAC1B,UAAU,CAAC,EAAE,UAAU,CAAC;CACzB,GAAG,OAAO,CAAC,SAAS,CAAC,CA2DrB;AAED,gFAAgF;AAChF,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,cAAc,EAAE,MAAM,CAAC;IACvB,YAAY,EAAE,MAAM,CAAC;CACtB;AAED,6FAA6F;AAC7F,wBAAgB,iBAAiB,CAAC,MAAM,EAAE,MAAM,GAAG,gBAAgB,EAAE,CAmCpE;AAED,iFAAiF;AACjF,eAAO,MAAM,eAAe,KAAK,CAAC;AAClC,wEAAwE;AACxE,eAAO,MAAM,sBAAsB,QAA0B,CAAC;AAE9D;;;;;;GAMG;AACH,wBAAgB,eAAe,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,EAAE;IAAE,KAAK,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAA;CAAE,GAAG,MAAM,EAAE,CAmCpH;AAMD,8GAA8G;AAC9G,eAAO,MAAM,4BAA4B,EAAE,SAAS,MAAM,EAA0B,CAAC;AACrF,eAAO,MAAM,qBAAqB,OAAQ,CAAC;AAC3C,oFAAoF;AACpF,eAAO,MAAM,mBAAmB,SAAU,CAAC;AAM3C,kDAAkD;AAClD,wBAAgB,iBAAiB,CAAC,QAAQ,EAAE,OAAO,GAAG,QAAQ,IAAI,MAAM,CAEvE;AAED,MAAM,WAAW,6BAA6B;IAC5C,wEAAwE;IACxE,UAAU,EAAE,MAAM,CAAC;IACnB,WAAW,EAAE,MAAM,CAAC;IACpB;;;;;;;;;;;;OAYG;IACH,QAAQ,EAAE,MAAM,CAAC;IACjB,UAAU,EAAE,CAAC,QAAQ,EAAE,gBAAgB,EAAE,KAAK,IAAI,CAAC;IACnD,MAAM,CAAC,EAAE,UAAU,CAAC;IACpB,GAAG,CAAC,EAAE,MAAM,MAAM,CAAC;IACnB,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,EAAE,OAAO,EAAE,MAAM,KAAK,IAAI,CAAC;CACzD;AAED;;;;;GAKG;AACH,qBAAa,sBAAsB;IACjC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAS;IAC9B,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAS;IACrC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAS;IAClC,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAyC;IACpE,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAa;IACpC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAe;IACnC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAoD;IACxE,OAAO,CAAC,MAAM,CAA2E;IACzF,OAAO,CAAC,KAAK,CAA+B;IAC5C,oEAAoE;IACpE,OAAO,CAAC,YAAY,CAAuB;IAC3C,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,OAAO,CAAS;IACxB,QAAQ,CAAC,KAAK;;;;;;MAAmE;gBAErE,OAAO,EAAE,6BAA6B;IAalD,qEAAqE;IACrE,KAAK,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI;
|
|
1
|
+
{"version":3,"file":"AlertClips.d.ts","sourceRoot":"","sources":["../src/AlertClips.ts"],"names":[],"mappings":"AAmCA,OAAO,EAAqB,KAAK,UAAU,EAAE,MAAM,yBAAyB,CAAC;AAC7E,OAAO,EAA0B,KAAK,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAEvF,+EAA+E;AAC/E,MAAM,WAAW,WAAW;IAC1B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,cAAc,EAAE,MAAM,CAAC;IACvB,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;CACtB;AAYD;;;GAGG;AACH,wBAAgB,gBAAgB,CAAC,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,WAAW,EAAE,CAmC/F;AAED;;;;;GAKG;AACH,wBAAgB,eAAe,CAAC,QAAQ,EAAE,WAAW,EAAE,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,WAAW,EAAE,CAYrH;AAOD,2DAA2D;AAC3D,wBAAgB,YAAY,CAAC,CAAC,EAAE;IAAE,QAAQ,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,MAAM,CAAC;IAAC,WAAW,EAAE,MAAM,CAAC;IAAC,GAAG,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,OAAO,CAAA;CAAE,GAAG,MAAM,EAAE,CAclI;AAED,MAAM,WAAW,OAAO;IACtB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,cAAc,EAAE,MAAM,CAAC;IACvB,YAAY,EAAE,MAAM,CAAC;IACrB,2CAA2C;IAC3C,cAAc,EAAE,MAAM,CAAC;CACxB;AAED,MAAM,MAAM,SAAS,GAAG;IAAE,IAAI,EAAE,OAAO,CAAA;CAAE,GAAG;IAAE,WAAW,EAAE,mBAAmB,GAAG,WAAW,GAAG,YAAY,CAAC;IAAC,MAAM,CAAC,EAAE,MAAM,CAAA;CAAE,CAAC;AAE/H,MAAM,MAAM,UAAU,GAAG,CACvB,IAAI,EAAE,MAAM,EACZ,IAAI,EAAE,MAAM,EAAE,EACd,OAAO,EAAE;IAAE,OAAO,EAAE,MAAM,CAAC;IAAC,WAAW,EAAE,OAAO,CAAA;CAAE,EAClD,QAAQ,EAAE,CAAC,KAAK,EAAE,KAAK,GAAG,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,KAAK,IAAI,KACpE,OAAO,CAAC;AAEb,+FAA+F;AAC/F,wBAAsB,cAAc,CAAC,CAAC,EAAE;IACtC,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,MAAM,EAAE,MAAM,CAAC;IACf,OAAO,EAAE,gBAAgB,CAAC;IAC1B,UAAU,CAAC,EAAE,UAAU,CAAC;CACzB,GAAG,OAAO,CAAC,SAAS,CAAC,CA2DrB;AAED,gFAAgF;AAChF,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,cAAc,EAAE,MAAM,CAAC;IACvB,YAAY,EAAE,MAAM,CAAC;CACtB;AAED,6FAA6F;AAC7F,wBAAgB,iBAAiB,CAAC,MAAM,EAAE,MAAM,GAAG,gBAAgB,EAAE,CAmCpE;AAED,iFAAiF;AACjF,eAAO,MAAM,eAAe,KAAK,CAAC;AAClC,wEAAwE;AACxE,eAAO,MAAM,sBAAsB,QAA0B,CAAC;AAE9D;;;;;;GAMG;AACH,wBAAgB,eAAe,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,EAAE;IAAE,KAAK,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAA;CAAE,GAAG,MAAM,EAAE,CAmCpH;AAMD,8GAA8G;AAC9G,eAAO,MAAM,4BAA4B,EAAE,SAAS,MAAM,EAA0B,CAAC;AACrF,eAAO,MAAM,qBAAqB,OAAQ,CAAC;AAC3C,oFAAoF;AACpF,eAAO,MAAM,mBAAmB,SAAU,CAAC;AAM3C,kDAAkD;AAClD,wBAAgB,iBAAiB,CAAC,QAAQ,EAAE,OAAO,GAAG,QAAQ,IAAI,MAAM,CAEvE;AAED,MAAM,WAAW,6BAA6B;IAC5C,wEAAwE;IACxE,UAAU,EAAE,MAAM,CAAC;IACnB,WAAW,EAAE,MAAM,CAAC;IACpB;;;;;;;;;;;;OAYG;IACH,QAAQ,EAAE,MAAM,CAAC;IACjB,UAAU,EAAE,CAAC,QAAQ,EAAE,gBAAgB,EAAE,KAAK,IAAI,CAAC;IACnD,MAAM,CAAC,EAAE,UAAU,CAAC;IACpB,GAAG,CAAC,EAAE,MAAM,MAAM,CAAC;IACnB,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,EAAE,OAAO,EAAE,MAAM,KAAK,IAAI,CAAC;CACzD;AAED;;;;;GAKG;AACH,qBAAa,sBAAsB;IACjC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAS;IAC9B,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAS;IACrC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAS;IAClC,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAyC;IACpE,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAa;IACpC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAe;IACnC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAoD;IACxE,OAAO,CAAC,MAAM,CAA2E;IACzF;;;;;;;;;OASG;IACH,OAAO,CAAC,eAAe,CAAuB;IAC9C,OAAO,CAAC,KAAK,CAA+B;IAC5C,oEAAoE;IACpE,OAAO,CAAC,YAAY,CAAuB;IAC3C,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,OAAO,CAAS;IACxB,QAAQ,CAAC,KAAK;;;;;;MAAmE;gBAErE,OAAO,EAAE,6BAA6B;IAalD,yEAAyE;IACzE,gBAAgB,IAAI,OAAO;IAS3B,qEAAqE;IACrE,KAAK,CAAC,UAAU,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE;QAAE,QAAQ,CAAC,EAAE,OAAO,CAAA;KAAE,GAAG,IAAI;IAe9D,IAAI,IAAI,IAAI;IASZ,OAAO,CAAC,QAAQ;YAWF,IAAI;CAwEnB"}
|