@ai-ecoverse/slicc-browsers 1.13.2 → 1.13.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -11
- package/guard-driver.mjs +31 -4
- package/guard-runner.mjs +125 -34
- package/guard-vectors.json +91 -7
- package/package.json +3 -3
- package/protected-error-page.mjs +52 -43
- package/redirect-secret.mjs +203 -51
package/README.md
CHANGED
|
@@ -190,9 +190,9 @@ The `C9` foreign extension isolated-world family requires `extensionWorld`: a br
|
|
|
190
190
|
|
|
191
191
|
The integration launcher loads shared-web 1.25.2’s harness extension exactly once through `Extensions.loadUnpacked` (`ankmbodmgimjemnojmeddfcdlbocolob`), whose content script runs on HTTP(S) pages. shared-web 1.25.2 also loads it by default in its own harness. A transport unable to host a foreign content-script extension omits `extensionWorld` and reports `not-supported:extensionWorld`; an advertised capability with no observed world fails, rather than silently skipping.
|
|
192
192
|
|
|
193
|
-
`C2.redirect-secret` probes ordinary fixture 302s to seven’s `/auth/callback?code=…&state=…` and vault’s `/oauth/mcp-callback?code=…`, plus `#access_token=…` variants of both. It navigates with Page and scripted `location.href`, fetches a subresource and sends an XHR. The 302 carries `Location`, `Content-Location`, `Refresh: 0;url=…` and `Link: <…>; rel=preconnect`. Page, Network (including ExtraInfo), Runtime, Log, Audits and both Fetch stages are enabled. Every received raw message is scanned, including replies, across an eight-second tail, for raw, percent-encoded, JSON-escaped and base64 marker representations. Both response-body APIs are attempted on each hop; guarded bodies need matched refusals. Page frame trees, resource trees and navigation history are read before recovery to catch error-page leftovers. A separate structural check requires every protected `Network.Request.urlFragment` and `Page.Frame.urlFragment` to be empty or absent, even if its value contains no marker; the ordinary `#access_token` control independently requires fragment evidence in request events, frame events, frame-tree replies and resource-tree replies. The ordinary session must survive. The unguarded ordinary redirect control preserves all four headers in `redirectResponse`, ExtraInfo’s headers and `headersText`, and Fetch’s response headers. `C2.redirect-secret-mix` repeats that matrix with 301/303/307/308. The protected leg requires in-place ExtraInfo delivery with origin-only header values, not merely the absence of a marker.
|
|
193
|
+
`C2.redirect-secret` probes ordinary fixture 302s to seven’s `/auth/callback?code=…&state=…` and vault’s `/oauth/mcp-callback?code=…`, plus `#access_token=…` variants of both. It navigates with Page and scripted `location.href`, fetches a subresource and sends an XHR. The 302 carries `Location`, `Content-Location`, `Refresh: 0;url=…` and `Link: <…>; rel=preconnect`. Page, Network (including ExtraInfo), Runtime, Log, Audits and both Fetch stages are enabled. Every received raw message is scanned, including replies but not the probe’s own observer session traffic, across an eight-second tail, for raw, percent-encoded, JSON-escaped and base64 marker representations. Both response-body APIs are attempted on each hop; guarded bodies need matched refusals. `Network.getResponseBody` is attempted, once per id, with every network id the client learns for a 301–308 hop: the `requestId` of a Network event carrying the redirect status and the `networkId` of the Fetch response-stage pause. For each registered hop whose id the client learned neither way, the probe reads the hop’s real id from the harness’s own unguarded tap (the observer session, which enables Network on the guarded leg) and attempts the read on the guarded session with it. The evidence’s `networkBodyCarriers` counts the ids per carrier, and `networkBodyPath` names the weakest carrier any id needed: `withheld+tap`, then `fetch-networkId`, then `network`, or `none` without any id. These reads never hold up a Fetch pause, and the probe gives them five seconds after `Fetch.disable` to settle. A read still unanswered then is not a refusal. `networkBodyAttempted` and `networkBodyRefused` hold only when every attempt is refused. On the guarded leg a `fetch-networkId` or `withheld+tap` carrier also needs the same row’s unguarded control to have seen the redirect’s Network events. Without that proof, or with no id at all, both read `incomplete` and the probe returns `redirect-secret: network body leg <reason>`; an attempt that is not refused is measured and fails. Every `Network.requestWillBeSent` the client sees for a registered redirect URL must end within the step in a redirect `requestWillBeSent`, `loadingFinished` or `loadingFailed` for that `requestId`. The evidence’s `hops` records per `requestId` the `delivery` (`redacted` for an origin-only protected redirect in place, `withheld` for a terminal without the redirect, `ordinary`, `raw` or `unterminated`) and the `terminal` event method. `requestsTerminated` is false on any `unterminated` request, a measured guard failure rather than missing evidence; `hopsRedacted` is false on any `raw` or `ordinary` hop, so a delivered protected hop must be origin-only, never the raw or a substituted URL; the guarded leg expects it. Page frame trees, resource trees and navigation history are read before recovery to catch error-page leftovers. A separate structural check requires every protected `Network.Request.urlFragment` and `Page.Frame.urlFragment` to be empty or absent, even if its value contains no marker; the ordinary `#access_token` control independently requires fragment evidence in request events, frame events, frame-tree replies and resource-tree replies. The ordinary session must survive. The unguarded ordinary redirect control preserves all four headers in `redirectResponse`, ExtraInfo’s headers and `headersText`, and Fetch’s response headers. `C2.redirect-secret-mix` repeats that matrix with 301/303/307/308. The protected leg requires in-place ExtraInfo delivery with origin-only header values, not merely the absence of a marker.
|
|
194
194
|
|
|
195
|
-
`C2.redirect-body` calls `Network.loadNetworkResource` on an ordinary redirect with `includeCredentials: true`, reads any stream with `IO.read`, replays a redirected XHR, requests Page resource content, and captures Page/MHTML and DOM snapshots after a main-frame redirect. It also captures an ordinary parent containing an iframe redirected to vault, including its error document, and overrides a paused ordinary Fetch request to the allowed redirect URL. No protected marker may appear in any message or body, and no request from those operations may send the independently seeded seven cookie. The ordinary control must return the body, resource content and snapshots, and accept the URL override. Each probe cleans up its redirect registrations, observer session and cookie. The focused Chrome tests use HTTPS with SPKI-pinned fixture trust and default-port protected URLs; host resolver rules route seven and vault to the fixture rather than the public service.
|
|
195
|
+
`C2.redirect-body` calls `Network.loadNetworkResource` on an ordinary redirect with `includeCredentials: true`, reads any stream with `IO.read`, replays a redirected XHR, requests Page resource content, and captures Page/MHTML and DOM snapshots after a main-frame redirect. It also captures an ordinary parent containing an iframe redirected to vault, including its error document, and overrides a paused ordinary Fetch request to the allowed redirect URL. No protected marker may appear in any message or body, and no request from those operations may send the independently seeded seven cookie. The ordinary control must return the body, resource content and snapshots, and accept the URL override. Its network-body carrier, `networkBodyAttempted`, `networkBodyRefused`, `hops`, `requestsTerminated` and `hopsRedacted` follow `C2.redirect-secret`. Each probe cleans up its redirect registrations, observer session and cookie. The focused Chrome tests use HTTPS with SPKI-pinned fixture trust and default-port protected URLs; host resolver rules route seven and vault to the fixture rather than the public service.
|
|
196
196
|
|
|
197
197
|
`C8.unparseable` probes both raw and canonical space/U+200D/invalid-punycode URLs through creation and navigation. Every guarded destination needs a matched refusal. Its unguarded controls record `document` or `native-rejection` outcomes, including Chrome builds that reject raw U+200D instead of committing it. Both raw and encoded-space forms must commit real fixture documents, because that is the proven cookie leak. An independent observer loads the encoded-space document and confirms it shares a uniquely seeded `.sliccy.ai` cookie; the guarded client must neither list nor attach to it, read its cookie, query that cookie by URL, nor cause a cookie-bearing destination request. The guard does not erase the observer’s private browser state. The row runs on both browser-session and page-session-only transports, uses the default context, and cleans up its own targets and cookie. Cookie seeding always uses the unguarded observer’s browser session, never the guarded client. Only guarded browser-session target listing requires `browserSession`; without it, `listed` records `not-supported` explicitly in probe and observation evidence, and the result lists that field in `notSupported`. Creation, navigation, attachment and `Network.getCookies` with `urls` still require matched refusals; filtering cookies without refusing fails `cookieRefused`. Fixture URLs omit the colon when the port is empty, and reported URLs are compared after canonicalisation, including Chrome’s WHATWG-unparseable encoded-space hosts. Launchers must resolve the raw/canonical space, U+200D/punycode and invalid-punycode `.sliccy.ai` fixture hosts to the fixture server, including for default-port HTTP or HTTPS origins. The HTTP fixture accepts Chrome’s encoded-space Host header without reparsing it as a WHATWG origin.
|
|
198
198
|
|
|
@@ -261,9 +261,9 @@ Popups are observed by `Target.targetCreated` with the opener's `openerId`, or b
|
|
|
261
261
|
|
|
262
262
|
`{ "extensionWorld": "hidden" | "select" | "debugger" | "drivable", "as": "world", "requires": ["extensionWorld"] }` runs the C9 compound probe on the current ordinary page. The runner attaches an independent observer to discover actual context ids, unique ids and names, and sends probes through the client session. Discovery matches the expected fixture extension’s exact origin, never the first extension world: `run()`’s optional `extensionWorldId` defaults to shared-web’s harness id `ankmbodmgimjemnojmeddfcdlbocolob`. Multiple unrelated content-script worlds are ignored for fixture selection, while guards must still protect foreign worlds. A custom fixture must provide the same content-script marker contract. Object handles are minted and validated on the unguarded control session, then replayed as guesses on the guarded leg; Chrome object handles and DOM node ids are session-local, so observer handles must never be mistaken for client handles. Numeric ids from the control are also replayed alongside fresh observer-discovered selectors, but numeric ids are local to a tab: a control id may name the next tab’s exposed default context, which must work and return no extension marker. The same-tab hidden world’s actual numeric id and every extension unique-id guess still require a matched refusal. Unknown numeric ids and stale object handles (`not exposed`) need explicit refusals rather than Chrome’s invalid-id errors. `guessesRefused` aggregates only required refusals; `guessesReached` must be false on the guarded leg, and `exposedContextsWork` must remain true. Its result contains boolean evidence consumed by ordinary `result` checks. The unguarded leg validates those selectors against Chrome itself before guarded evidence can pass.
|
|
263
263
|
|
|
264
|
-
`{ "errorPage": "recover", "secret"?: "all" | "query" | "fragment", "as": "errorPage" }` (also modes `restore` and `sticky`; `secret` defaults to `all`) runs the protected-error-page compound probe with independent commit evidence, discovery redaction and recovery or sticky-lock checks. The unguarded observer seeds the protected history and failed commit, so the rows run on page-session-only transports too. The guarded client uses only `Target.getTargetInfo` with a `targetId`, `Target.getTargets`, `Target.setDiscoverTargets`, `Target.attachToTarget`, `Target.detachFromTarget` and `Target.closeTarget` without a session, which page-session transports answer themselves, and everything else on its page session; the rows require no whole-vector capability. Only `eventsRedacted` requires `discovery`: without it the probe does not call `Target.setDiscoverTargets`, that one field is `not-supported` in both legs, and the notes say `not-supported:discovery`; info, list, attach, recovery and lock checks still run. Discovery absence is structural: a guarded transport delivering either `Target.targetCreated` or `Target.targetInfoChanged` during such a row without declaring `discovery` fails the row, even when the event names an ordinary target. `Target.attachedToTarget` remains independently checked by `attachRedacted`. The runner leaves this probe’s independent Fetch sessions to the probe rather than answering their pauses itself.
|
|
264
|
+
`{ "errorPage": "recover", "secret"?: "all" | "query" | "fragment", "as": "errorPage" }` (also modes `restore` and `sticky`; `secret` defaults to `all`) runs the protected-error-page compound probe with independent commit evidence, discovery redaction and recovery or sticky-lock checks. The unguarded observer seeds the protected history and failed commit, so the rows run on page-session-only transports too. The guarded client uses only `Target.getTargetInfo` with a `targetId`, `Target.getTargets`, `Target.setDiscoverTargets`, `Target.attachToTarget`, `Target.detachFromTarget` and `Target.closeTarget` without a session, which page-session transports answer themselves, and everything else on its page session; the rows require no whole-vector capability. Only `eventsRedacted` requires `discovery`: without it the probe does not call `Target.setDiscoverTargets`, that one field is `not-supported` in both legs, and the notes say `not-supported:discovery`; info, list, attach, recovery and lock checks still run. Discovery absence is structural: a guarded transport delivering either `Target.targetCreated` or `Target.targetInfoChanged` during such a row without declaring `discovery` fails the row, even when the event names an ordinary target. `Target.attachedToTarget` remains independently checked by `attachRedacted`. The runner leaves this probe’s independent Fetch sessions to the probe rather than answering their pauses itself. Each of the probe’s waits is bounded at 5000 ms, scaled by `settleScale`. When the probe’s own evidence is missing, it returns `incomplete` with `protected-error-page: <reason>` instead of throwing: the observer target could not be created or attached (an `openerId` included, after cleaning the target up), the observer’s history seed or allowed document did not commit (or its setup failed), the control’s own Fetch failure injection left no error commit, the protected history entry is missing, no real `BackForwardCacheRestore` arrived, or the observer’s sticky failure left no error page. The guarded leg’s missing error commit is not incomplete: it is the guard’s, and stays `live: false`.
|
|
265
265
|
|
|
266
|
-
`{ "redirectSecret": "headers" | "mix" | "body", "as": "redirect" }` runs a C2 compound probe and owns its Fetch pauses. The client must expose `messages`, an append-only array of every raw received CDP text frame, recorded before parsing; an events-only or reply-only tap fails `rawRecorded`. Whenever a run includes a `C2.redirect-*` row or a `redirectSecret` step, `run()` retains an explicit `fixture.redirectSecret` hook or automatically installs a fetch-only `POST /redirect` bridge. Its base is `fixtureBase`, then `fixture.origin`, then `targets.ordinary`; a browser-page alias or reverse proxy must forward that endpoint to the bundled fixture. The bundled `fixtureServer()` always supplies both its direct hook and HTTP control endpoint, so no custom registration method is needed. Missing, unreachable or non-successful registration is an explicit row/summary error containing `redirectSecret fixture registration required`, never a pass or a skipped observation. Fixture logs must include headers for the cookie check. The guard’s events and raw messages must represent the same delivered stream.
|
|
266
|
+
`{ "redirectSecret": "headers" | "mix" | "body", "as": "redirect" }` runs a C2 compound probe and owns its Fetch pauses. The client must expose `messages`, an append-only array of every raw received CDP text frame, recorded before parsing; an events-only or reply-only tap fails `rawRecorded`. Whenever a run includes a `C2.redirect-*` row or a `redirectSecret` step, `run()` retains an explicit `fixture.redirectSecret` hook or automatically installs a fetch-only `POST /redirect` bridge. Its base is `fixtureBase`, then `fixture.origin`, then `targets.ordinary`; a browser-page alias or reverse proxy must forward that endpoint to the bundled fixture. The bundled `fixtureServer()` always supplies both its direct hook and HTTP control endpoint, so no custom registration method is needed. Missing, unreachable or non-successful registration is an explicit row/summary error containing `redirectSecret fixture registration required`, never a pass or a skipped observation. Fixture logs must include headers for the cookie check. The guard’s events and raw messages must represent the same delivered stream. The leak scan memoizes each decoded string and yields to the event loop between items, so concurrent rows in the same process are not starved. When the probe’s own unguarded observer setup (attach, `Page.enable` and the seven cookie) fails before the client is used, the probe returns `incomplete` with `redirect-secret: observer setup failed: <reason>` instead of throwing.
|
|
267
267
|
|
|
268
268
|
**Host-harness requirements for every guard.** Host Node processes may use `fixtureServer()` from `./fixture`; browser pages must import only the browser-safe runner graph, served with JavaScript MIME types, and need no Node-global shim. All guards supply a guarded client, an independent unguarded observer, mapped fixture origins and the relevant advertised capabilities. C2 clients additionally record every received raw CDP frame in `client.messages` before parsing. Route protected seven/vault hosts to the fixture with `--host-resolver-rules`, including default-port callback URLs; a live protected-body negative needs HTTPS fixture trust (the reference harness pins its SPKI).
|
|
269
269
|
|
|
@@ -285,7 +285,7 @@ For remote/page harnesses, `fixtureClient(base)` is exported by `./guard-runner`
|
|
|
285
285
|
|
|
286
286
|
**Variants.** `variants` gives the delay in milliseconds before the client's `Target.detachFromTarget`: `prompt` 0, `lagging` 8000. A runner may add its own variant names.
|
|
287
287
|
|
|
288
|
-
**Reports.** There is one row per id, target, variant and transport. `control`, `guarded` and `live` are each `pass`, `fail`, `skip` (with the skip's `reason`) or `not-run` (for text rows):
|
|
288
|
+
**Reports.** There is one row per id, target, variant and transport. `control`, `guarded` and `live` are each `pass`, `fail`, `incomplete` (missing probe evidence, listed in the row's `incomplete`), `skip` (with the skip's `reason`) or `not-run` (for text rows):
|
|
289
289
|
|
|
290
290
|
```json
|
|
291
291
|
{ "id": "C1.script-nav", "target": "vault", "variant": "lagging", "transport": "cli", "via": "guard", "control": "pass", "guarded": "pass", "live": "pass" }
|
|
@@ -303,6 +303,7 @@ It exports:
|
|
|
303
303
|
- `refusals(vector)`: the vector's refusal names;
|
|
304
304
|
- `outcome`, `call(client, step, sessionId)` and `isRefusal(outcome, step)`;
|
|
305
305
|
- `instrument(client, sessionId, log, tag)`, `load(client, sessionId, url)` and `popup(client, openerId)`;
|
|
306
|
+
- `waitFor(check, ms, step?)`: polls `check` every `step` ms (default 25) until it holds, checking once more after the bound. A timer heartbeat every `step` runs for the whole wait, including while `check` is awaiting a reply, and any gap between beats longer than a second (or four steps, if longer) extends the bound by that gap, so a synchronous event-loop stall elsewhere in the process does not consume it. The extensions add up to at most `max(4 × ms, 30000)` ms, so a throttled page cannot stretch a wait without limit. `load`, the runner's waits and the error-page and redirect-body probes use it;
|
|
306
307
|
- `judge(vector, side, observed)`: `{ pass, refused, skipped, mismatches }`, where `refused` lists the leg's refusals and `skipped` the observations that depend on them or explicitly record unavailable observation capabilities;
|
|
307
308
|
- `result(vector, run)`: a report row from `{ target, variant, transport, via?, capabilities?, control, guarded, live }`. `capabilities` is the transport's; without it, the transport has them all;
|
|
308
309
|
- `parseURL(value, base?)`: a `URL`, or `null` for anything `new URL` rejects or a non-string; every parse of a fixture log entry or a Chrome-reported URL in the runner and the fixture goes through it (`protected.mjs` keeps its own copy, since it imports nothing), so an unparseable entry never throws;
|
|
@@ -327,7 +328,7 @@ It exports:
|
|
|
327
328
|
`guard-runner.mjs` is dependency-free browser ESM. `run(set, options)` returns
|
|
328
329
|
`{ results, summary(metadata) }`; the module also exports `summary`. Results have
|
|
329
330
|
`id`, `target`, `variant`, `control`, `guarded`, `live`, boolean observations and
|
|
330
|
-
errors. Statuses are `pass`, `fail`, `skip` and `not-run`. Text vectors are reported
|
|
331
|
+
errors. Statuses are `pass`, `fail`, `incomplete`, `skip` and `not-run`. Text vectors are reported
|
|
331
332
|
as `not-run`; they are not silently certified. A missing observation check is an
|
|
332
333
|
error before execution.
|
|
333
334
|
|
|
@@ -348,7 +349,7 @@ The factory can configure the guard's own transport delay from this metadata;
|
|
|
348
349
|
the runner never reaches into the guard. `transportVariant: true` acknowledges
|
|
349
350
|
that configuration, and every result records that boolean (false by default).
|
|
350
351
|
`slot` allows pooling browsers while keeping each row's clients independent.
|
|
351
|
-
The optional `close` releases the row's clients; release pooled browsers after
|
|
352
|
+
A factory that rejects or throws makes only that row `incomplete` for control, guarded and live, with `clientFactory: <reason>` in its `incomplete`; the run goes on with the next row instead of rejecting. Such a row is labelled with the transport of the last client the factory made (`cdp` if none), and is passed to `log` once the run completes, so `summary()` never counts a placeholder transport. The optional `close` releases the row's clients; release pooled browsers after
|
|
352
353
|
`run` completes. Each parallel slot needs an independent browser for
|
|
353
354
|
browser-wide settings. Without a factory execution is serial. Cookie and storage
|
|
354
355
|
rows use fresh browser contexts when `capabilities.browserContexts` is true;
|
|
@@ -380,12 +381,14 @@ every alternative of an `any`, or a required (`all`, `not` or plain) check, need
|
|
|
380
381
|
selects one variant or an array; omission selects all. `parallel` defaults to 1.
|
|
381
382
|
With the `browserContexts` capability each leg runs in its own browser context. A `Target.createBrowserContext` that fails with a timeout is retried once, and the row's `notes` records it; any other failure fails the row.
|
|
382
383
|
|
|
383
|
-
Every target created by the runner’s own unguarded observer uses `Target.createTarget` with `newWindow: true`, then verifies through `Target.getTargetInfo` that `openerId` is absent. A supplied opener or missing target evidence fails and cleans up that target, rather than inheriting the active client tab (which could be seven). This includes ordinary control setup, storage canaries, protected-error history seeds and the malformed-URL observer. The row notes record `newWindow=true, openerId absent`. Intentional guarded-client creation commands and popup/opener tests keep their original parameters.
|
|
384
|
+
Every target created by the runner’s own unguarded observer uses `Target.createTarget` with `newWindow: true`, then verifies through `Target.getTargetInfo` that `openerId` is absent. A supplied opener or missing target evidence fails and cleans up that target (in the protected-error-page probe, the row is `incomplete` instead), rather than inheriting the active client tab (which could be seven). This includes ordinary control setup, storage canaries, protected-error history seeds and the malformed-URL observer. The row notes record `newWindow=true, openerId absent`. Intentional guarded-client creation commands and popup/opener tests keep their original parameters.
|
|
384
385
|
|
|
385
386
|
`settleScale` (default 1) multiplies the bounds of waits for events the leg expects to happen: settles (beacon, served, title and load) on the control leg and `required` settles on either leg, `paused` on the control leg, page and child loads, the wait for a background step's `as` result (`<as> did not complete within <ms>ms`), the wait for the pre-action beacons, the C9 control's `Debugger.paused`, and the wait for an observation the leg expects to be `true`. Adopters on slow hosts can set 3 or 4. Every wait polls and returns as soon as its event arrives, so a larger bound only costs time when something is really missing, and each timeout error names the bound it used. An observation read from the fixture (`observer: "fixture"`, such as a beacon or a served request) that the leg expects to be `true` is polled for up to 5000 ms (scaled) instead of read once. One that the leg expects to be `false` is still read once.
|
|
386
387
|
|
|
387
388
|
`refusalScale` (default 1) multiplies the bounds of waits for what the guarded leg expects to be absent, the beacon, request, title or navigation that the guard should block: settles that are not `required`, and `paused`, on the guarded leg. Such a wait runs to its full bound whenever the guard works, so `settleScale` leaves it alone. The trade-off: a larger `refusalScale` gives more sensitivity to a leak that arrives late on a slow host, because the observations after the wait see it, at the cost of that much more time per guarded row. Adopters should keep `refusalScale` at 1 or higher, never below 1; the runner refuses a `refusalScale` below 1 with `refusalScale must be a number of at least 1`.
|
|
388
389
|
|
|
390
|
+
**Incomplete.** A probe that returns `incomplete` (only `errorPage` and `redirectSecret` steps) records the reason in the row's `incomplete` as `<leg> <probe>: <reason>`. Its unmeasured observations read `"incomplete"`, and so does guarded liveness when the probe never measured it. A leg whose measured observations all match is then `incomplete`, never `pass`, and `live` is `incomplete` when only its evidence is missing; a measured mismatch still makes the leg `fail`, and any row error still fails both legs. `summary()` counts `incomplete` per leg, returns the number of rows with any `incomplete` status as `incomplete`, appends `incomplete <n>` to `line`, and never passes such a run. A leg that throws keeps the errors it recorded before the throw.
|
|
391
|
+
|
|
389
392
|
`onStart(row)` receives each row's `id`, `target`, `variant`, `transport` and `via` as the row starts. `run()` returns a promise that also has `inFlight()`, and its result has `inFlight` too. `inFlight()` lists the rows that are running now, each as `{ id, target, variant, leg, phase, step, clients }`, where `step` is the index of the current step in its `phase` (`setup`, `trigger` or `child`), and `clients` lists each client's pending CDP calls as `{ method, session, ageMs }` (`client`, plus `unguarded` when it is a separate client). With `rowTimeoutMs`, a row still running after that many milliseconds fails on its own. Its `errors` hold `row timed out after <ms>ms: <the in-flight view>`, `inFlight` holds the view, and the clients that `clientFactory` made for it are closed. The run then goes on to the next row. It is off by default, and a client passed as `client` is never closed.
|
|
390
393
|
|
|
391
394
|
`log(row)` receives completed rows. Besides `control`, `guarded`, `live`, `why` and `errors`, a row may carry `notSupported` (unavailable observation fields), `ran` (steps that ran on one leg only) and `notes` (settles skipped because their registration was refused). Both are informational: they explain a row's verdict, and adopters (bios, slicc-cli, slicc-cdp) must not count them as failures, or as passes. The verdict is `control`, `guarded` and `live`. A variant supplies the wait before an
|
|
@@ -640,15 +643,15 @@ Changing a table in `slicc-cli/internal/browser` means changing `catalog.json` h
|
|
|
640
643
|
|
|
641
644
|
## Releases
|
|
642
645
|
|
|
643
|
-
semantic-release cuts one version from `main` for all three packages (`.github/workflows/release.yml`)
|
|
646
|
+
semantic-release cuts one version from `main` for all three packages (`.github/workflows/release.yml`). The maintenance branch `1.13.x` (`.releaserc.json`: range `1.13.x`, channel `release-1.13.x`) cuts `1.13.N` patches the same way; it takes only `fix:` commits, cherry-picked from `main` after they merge there, through pull requests that merge on green CI.
|
|
644
647
|
1. The tag `vX.Y.Z` and a GitHub release are the Go module's release. `scripts/major-guard.mjs` fails the release when the major doesn't fit the module path in `go.mod` (no suffix allows v0 and v1, `/vN` only vN). The workflow then asks `proxy.golang.org` for the new version and `pkg.go.dev` to fetch it; both only warn.
|
|
645
|
-
2. **npm:** with the repository variable `SLICC_BROWSERS_NPM_PUBLISH` set to `true`, the package is stamped with the version and published through npm trusted publishing (OIDC, with provenance). A version that is already on npm is skipped.
|
|
648
|
+
2. **npm:** with the repository variable `SLICC_BROWSERS_NPM_PUBLISH` set to `true`, the package is stamped with the version and published through npm trusted publishing (OIDC, with provenance). A version that is already on npm is skipped. `scripts/dist-tag.mjs <version> <ref> <latest>` picks the dist-tag: `latest` for a run from `main` whose version is not below the current `latest`, otherwise `release-X.Y.x`; a run from `main` fails when it cannot read `latest`. So a `1.13.x` release publishes to `release-1.13.x` and never moves `latest`. The GitHub release of a maintenance version is not marked latest, and the Go proxy and crates.io order by version.
|
|
646
649
|
3. **crates.io:** with the repository variable `SLICC_BROWSERS_CRATES_PUBLISH` set to `true`, `Cargo.toml` is stamped with the version and `cargo publish` runs with a short-lived token from crates.io trusted publishing (`rust-lang/crates-io-auth-action`). A version that is already on crates.io is skipped.
|
|
647
650
|
4. **Cert line:** after the npm publish, the workflow runs `npm pack @ai-ecoverse/slicc-browsers@X.Y.Z` against the registry and appends `cert: @ai-ecoverse/slicc-browsers@X.Y.Z sha256=<hex> gitHead=<sha>` to the `vX.Y.Z` GitHub release and the job summary, once; the cert lane copies it from there.
|
|
648
651
|
|
|
649
652
|
GitHub sometimes rejects the push of `refs/notes/semantic-release-vX.Y.Z` with a bare `(failed)` right after the tag push; no ruleset covers `refs/notes/*`. The semantic-release step runs with `pipefail` and puts `scripts/git-retry.mjs` first on `PATH` as `git`: it runs the real git (`REAL_GIT`) and retries a push of a `refs/notes/` ref up to five times, 5, 10, 15 and 20 seconds apart; any other git command runs once. When semantic-release still fails, the job fails before npm and crates.io publish. Recover without moving the tag: create the GitHub release on the existing tag, then run `gh workflow run release.yml --ref main -f tag=vX.Y.Z`. A tag without its note counts as on the default channel (semantic-release reads a missing note as `channels: [null]`), so the next release builds on it. `test/integration/git-retry.test.mjs` tests the retry with a fake git.
|
|
650
653
|
|
|
651
|
-
A run started by hand publishes what's missing without a new release: `gh workflow run release.yml --ref main` when `main`'s head has its tag, or `gh workflow run release.yml --ref main -f tag=vX.Y.Z` for any existing tag. `gh workflow run release.yml --ref main -f cert-only=true` only adds the cert line for the latest npm version.
|
|
654
|
+
A run started by hand publishes what's missing without a new release: `gh workflow run release.yml --ref main` when `main`'s head has its tag, or `gh workflow run release.yml --ref main -f tag=vX.Y.Z` for any existing tag. `gh workflow run release.yml --ref main -f cert-only=true` only adds the cert line for the latest npm version. On the maintenance line, use `--ref 1.13.x -f tag=v1.13.N`.
|
|
652
655
|
|
|
653
656
|
**Go module versions.** A pushed tag is the release: `proxy.golang.org` caches it and `sum.golang.org` records its hash for good. Never move or delete a pushed tag; retract a bad version in `go.mod` and release anew. A `BREAKING CHANGE` needs a `/vN` module path first.
|
|
654
657
|
|
package/guard-driver.mjs
CHANGED
|
@@ -111,10 +111,37 @@ export async function load(client, sessionId, url, ms = 20000) {
|
|
|
111
111
|
const start = client.events.length;
|
|
112
112
|
const { loaderId } = await client.send('Page.navigate', { url }, sessionId);
|
|
113
113
|
if (!loaderId) return;
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
114
|
+
if (!(await waitFor(() => loadedAfter(client.events.slice(start), sessionId, loaderId), ms)))
|
|
115
|
+
throw new Error(`load of ${url} timed out`);
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
export async function waitFor(check, ms, step = 25) {
|
|
119
|
+
const start = Date.now();
|
|
120
|
+
const limit = start + ms + Math.max(ms * 4, 30000);
|
|
121
|
+
let deadline = start + ms;
|
|
122
|
+
let last = start;
|
|
123
|
+
const beat = () => {
|
|
124
|
+
const now = Date.now();
|
|
125
|
+
const gap = now - last;
|
|
126
|
+
if (gap > Math.max(step * 4, 1000)) deadline = Math.min(limit, deadline + gap - step);
|
|
127
|
+
last = now;
|
|
128
|
+
};
|
|
129
|
+
let timer;
|
|
130
|
+
const tick = () => {
|
|
131
|
+
beat();
|
|
132
|
+
timer = setTimeout(tick, step);
|
|
133
|
+
};
|
|
134
|
+
timer = setTimeout(tick, step);
|
|
135
|
+
try {
|
|
136
|
+
for (;;) {
|
|
137
|
+
if (await check()) return true;
|
|
138
|
+
beat();
|
|
139
|
+
if (Date.now() > deadline) return false;
|
|
140
|
+
await new Promise((resolve) => setTimeout(resolve, step));
|
|
141
|
+
beat();
|
|
142
|
+
}
|
|
143
|
+
} finally {
|
|
144
|
+
clearTimeout(timer);
|
|
118
145
|
}
|
|
119
146
|
}
|
|
120
147
|
|
package/guard-runner.mjs
CHANGED
|
@@ -16,6 +16,7 @@ import {
|
|
|
16
16
|
summary as report,
|
|
17
17
|
result,
|
|
18
18
|
steps,
|
|
19
|
+
waitFor,
|
|
19
20
|
} from './guard-driver.mjs';
|
|
20
21
|
import { createObserverTarget, discoveryViolation } from './observer-target.mjs';
|
|
21
22
|
import { protectedErrorPage } from './protected-error-page.mjs';
|
|
@@ -120,6 +121,7 @@ class Execution {
|
|
|
120
121
|
this.contexts = [];
|
|
121
122
|
this.pending = [];
|
|
122
123
|
this.errors = [];
|
|
124
|
+
this.incomplete = [];
|
|
123
125
|
this.preSent = [];
|
|
124
126
|
this.notes = [];
|
|
125
127
|
this.answers = [];
|
|
@@ -193,12 +195,7 @@ class Execution {
|
|
|
193
195
|
}
|
|
194
196
|
|
|
195
197
|
async poll(predicate, ms = 1500, absent = false) {
|
|
196
|
-
|
|
197
|
-
do {
|
|
198
|
-
if (await predicate()) return true;
|
|
199
|
-
await sleep(25);
|
|
200
|
-
} while (Date.now() < deadline);
|
|
201
|
-
return false;
|
|
198
|
+
return waitFor(predicate, this.bound(ms, absent));
|
|
202
199
|
}
|
|
203
200
|
|
|
204
201
|
async prepareState(spec) {
|
|
@@ -641,6 +638,8 @@ class Execution {
|
|
|
641
638
|
(key) => step[key]
|
|
642
639
|
);
|
|
643
640
|
if (probe && !out.ok) this.errors.push(`${probe}: ${out.error?.message ?? 'probe failed'}`);
|
|
641
|
+
if (['errorPage', 'redirectSecret'].includes(probe) && out.ok && out.result?.incomplete)
|
|
642
|
+
this.incomplete.push(`${probe}: ${out.result.incomplete}`);
|
|
644
643
|
if (step.as && !out.result?.pending) this.calls[step.as] = out;
|
|
645
644
|
if (step.cdp) (this.calls.$cdp ??= []).push({ method: step.cdp, ...out });
|
|
646
645
|
}
|
|
@@ -704,6 +703,7 @@ class Execution {
|
|
|
704
703
|
fixture: this.options.fixture,
|
|
705
704
|
mode: step.redirectSecret,
|
|
706
705
|
side: this.side,
|
|
706
|
+
controls: (this.clients.redirectSecretControls ??= {}),
|
|
707
707
|
})
|
|
708
708
|
);
|
|
709
709
|
} finally {
|
|
@@ -726,6 +726,7 @@ class Execution {
|
|
|
726
726
|
secret: step.secret,
|
|
727
727
|
discovery: supported(['discovery'], this.options.capabilities),
|
|
728
728
|
notes: this.notes,
|
|
729
|
+
budget: this.bound(5000),
|
|
729
730
|
})
|
|
730
731
|
);
|
|
731
732
|
if (step.unparseable)
|
|
@@ -956,7 +957,10 @@ class Execution {
|
|
|
956
957
|
const want = (this.side === 'control' ? this.vector.control.expected : this.vector.expected)?.[
|
|
957
958
|
name
|
|
958
959
|
];
|
|
959
|
-
if (
|
|
960
|
+
if (
|
|
961
|
+
check?.result &&
|
|
962
|
+
(at(this.calls[check.result], check.path) === 'incomplete' || this.unmeasured(check))
|
|
963
|
+
)
|
|
960
964
|
return 'incomplete';
|
|
961
965
|
if (want !== true || check?.observer !== 'fixture') return this.check(check);
|
|
962
966
|
let seen = false;
|
|
@@ -967,6 +971,11 @@ class Execution {
|
|
|
967
971
|
return seen;
|
|
968
972
|
}
|
|
969
973
|
|
|
974
|
+
unmeasured(check) {
|
|
975
|
+
const out = check?.result ? this.calls[check.result] : undefined;
|
|
976
|
+
return Boolean(out?.result?.incomplete) && at(out, check.path) === undefined;
|
|
977
|
+
}
|
|
978
|
+
|
|
970
979
|
async check(check) {
|
|
971
980
|
const c = expand(check, this.vars);
|
|
972
981
|
if (!(await this.ready(c))) return false;
|
|
@@ -1186,6 +1195,8 @@ class Execution {
|
|
|
1186
1195
|
if (this.side === 'control') return { live: true };
|
|
1187
1196
|
const alive = () => this.check(this.vector.liveCheck);
|
|
1188
1197
|
let live = await alive();
|
|
1198
|
+
if (!live && this.unmeasured(this.vector.liveCheck))
|
|
1199
|
+
return { live: 'incomplete', liveWhy: 'incomplete' };
|
|
1189
1200
|
if (!live && !unseen.length) live = await this.poll(alive, 1500);
|
|
1190
1201
|
if (!unseen.length || (live && !incomplete.length)) return { live };
|
|
1191
1202
|
const detail = incomplete.length ? ` (${incomplete.join(', ')} incomplete)` : '';
|
|
@@ -1216,7 +1227,14 @@ class Execution {
|
|
|
1216
1227
|
side: this.side,
|
|
1217
1228
|
});
|
|
1218
1229
|
const { live, liveWhy } = await this.liveness(unseen, incomplete);
|
|
1219
|
-
return {
|
|
1230
|
+
return {
|
|
1231
|
+
observed,
|
|
1232
|
+
live,
|
|
1233
|
+
liveWhy,
|
|
1234
|
+
errors: this.errors,
|
|
1235
|
+
notes: this.notes,
|
|
1236
|
+
incomplete: this.incomplete,
|
|
1237
|
+
};
|
|
1220
1238
|
} finally {
|
|
1221
1239
|
this.stopped = true;
|
|
1222
1240
|
await pumping;
|
|
@@ -1517,18 +1535,17 @@ async function executeRow(job, opts, made, set, slot, index, flights) {
|
|
|
1517
1535
|
opts.onStart?.(base);
|
|
1518
1536
|
const sides = (async () => {
|
|
1519
1537
|
for (const side of ['control', 'guarded']) {
|
|
1538
|
+
const vars = variables(opts.targets, target, side, slot, index);
|
|
1539
|
+
const execution = new Execution(vector, side, opts, clients, vars, set.variants[variant]);
|
|
1520
1540
|
try {
|
|
1521
|
-
|
|
1522
|
-
legs[side] = await new Execution(
|
|
1523
|
-
vector,
|
|
1524
|
-
side,
|
|
1525
|
-
opts,
|
|
1526
|
-
clients,
|
|
1527
|
-
vars,
|
|
1528
|
-
set.variants[variant]
|
|
1529
|
-
).run();
|
|
1541
|
+
legs[side] = await execution.run();
|
|
1530
1542
|
} catch (error) {
|
|
1531
|
-
legs[side] = {
|
|
1543
|
+
legs[side] = {
|
|
1544
|
+
observed: {},
|
|
1545
|
+
live: false,
|
|
1546
|
+
errors: [...execution.errors, String(error.message ?? error)],
|
|
1547
|
+
notes: execution.notes,
|
|
1548
|
+
};
|
|
1532
1549
|
}
|
|
1533
1550
|
}
|
|
1534
1551
|
})();
|
|
@@ -1559,6 +1576,7 @@ async function executeRow(job, opts, made, set, slot, index, flights) {
|
|
|
1559
1576
|
...(legs.guarded.notes ?? []).map((note) => `guarded ${note}`),
|
|
1560
1577
|
];
|
|
1561
1578
|
if (notes.length) row.notes = notes;
|
|
1579
|
+
settleIncomplete(row, vector, base, legs);
|
|
1562
1580
|
if (row.errors.length) {
|
|
1563
1581
|
row.control = 'fail';
|
|
1564
1582
|
row.guarded = 'fail';
|
|
@@ -1567,6 +1585,37 @@ async function executeRow(job, opts, made, set, slot, index, flights) {
|
|
|
1567
1585
|
return row;
|
|
1568
1586
|
}
|
|
1569
1587
|
|
|
1588
|
+
function settleIncomplete(row, vector, base, legs) {
|
|
1589
|
+
const incomplete = ['control', 'guarded'].flatMap((side) =>
|
|
1590
|
+
(legs[side].incomplete ?? []).map((reason) => `${side} ${reason}`)
|
|
1591
|
+
);
|
|
1592
|
+
if (!incomplete.length) return;
|
|
1593
|
+
row.incomplete = incomplete;
|
|
1594
|
+
if (row.errors.length) return;
|
|
1595
|
+
const fill = (side) => {
|
|
1596
|
+
const want = side === 'control' ? vector.control.expected : vector.expected;
|
|
1597
|
+
return Object.fromEntries(
|
|
1598
|
+
Object.entries(legs[side].observed).map(([name, value]) => [
|
|
1599
|
+
name,
|
|
1600
|
+
value === 'incomplete' && Object.hasOwn(want, name)
|
|
1601
|
+
? want[name] === 'either' || want[name]
|
|
1602
|
+
: value,
|
|
1603
|
+
])
|
|
1604
|
+
);
|
|
1605
|
+
};
|
|
1606
|
+
const filled = result(vector, {
|
|
1607
|
+
...base,
|
|
1608
|
+
control: fill('control'),
|
|
1609
|
+
guarded: fill('guarded'),
|
|
1610
|
+
live: legs.control.live === true && [true, 'incomplete'].includes(legs.guarded.live),
|
|
1611
|
+
});
|
|
1612
|
+
for (const side of ['control', 'guarded'])
|
|
1613
|
+
if (legs[side].incomplete?.length && row[side] === 'fail' && filled[side] === 'pass')
|
|
1614
|
+
row[side] = 'incomplete';
|
|
1615
|
+
if (legs.guarded.live === 'incomplete' && filled.live === 'pass') row.live = 'incomplete';
|
|
1616
|
+
row.why = [filled.why, `incomplete: ${incomplete.join('; ')}`].filter(Boolean).join('; ');
|
|
1617
|
+
}
|
|
1618
|
+
|
|
1570
1619
|
const invalidBlocked = (check) =>
|
|
1571
1620
|
Boolean(
|
|
1572
1621
|
check?.blocked &&
|
|
@@ -1628,6 +1677,38 @@ function validate(set, opts) {
|
|
|
1628
1677
|
return variants;
|
|
1629
1678
|
}
|
|
1630
1679
|
|
|
1680
|
+
function factoryClients(opts, set, job, slot) {
|
|
1681
|
+
return outcome(
|
|
1682
|
+
(async () =>
|
|
1683
|
+
opts.clientFactory
|
|
1684
|
+
? opts.clientFactory({
|
|
1685
|
+
slot,
|
|
1686
|
+
variant: job.variant,
|
|
1687
|
+
detachDelay: set.variants[job.variant],
|
|
1688
|
+
vector: job.vector,
|
|
1689
|
+
target: job.target,
|
|
1690
|
+
})
|
|
1691
|
+
: opts.client)()
|
|
1692
|
+
);
|
|
1693
|
+
}
|
|
1694
|
+
|
|
1695
|
+
function unmade(job, message) {
|
|
1696
|
+
const reason = `clientFactory: ${message}`;
|
|
1697
|
+
return {
|
|
1698
|
+
id: job.vector.id,
|
|
1699
|
+
target: job.target,
|
|
1700
|
+
variant: job.variant,
|
|
1701
|
+
via: job.vector.control.via,
|
|
1702
|
+
transportVariant: false,
|
|
1703
|
+
control: 'incomplete',
|
|
1704
|
+
guarded: 'incomplete',
|
|
1705
|
+
live: 'incomplete',
|
|
1706
|
+
why: `incomplete: ${reason}`,
|
|
1707
|
+
errors: [],
|
|
1708
|
+
incomplete: [reason],
|
|
1709
|
+
};
|
|
1710
|
+
}
|
|
1711
|
+
|
|
1631
1712
|
export function run(set, options) {
|
|
1632
1713
|
const flights = new Set();
|
|
1633
1714
|
const inFlight = () =>
|
|
@@ -1655,20 +1736,21 @@ async function runAll(set, options, flights) {
|
|
|
1655
1736
|
const results = new Array(jobs.length);
|
|
1656
1737
|
let next = 0;
|
|
1657
1738
|
let serial = Promise.resolve();
|
|
1739
|
+
let transport = 'cdp';
|
|
1740
|
+
const unmadeRows = [];
|
|
1658
1741
|
const worker = async (slot) => {
|
|
1659
1742
|
for (;;) {
|
|
1660
1743
|
const index = next++;
|
|
1661
1744
|
if (index >= jobs.length) break;
|
|
1662
1745
|
const job = jobs[index];
|
|
1663
|
-
const
|
|
1664
|
-
|
|
1665
|
-
|
|
1666
|
-
|
|
1667
|
-
|
|
1668
|
-
|
|
1669
|
-
|
|
1670
|
-
|
|
1671
|
-
: opts.client;
|
|
1746
|
+
const factory = await factoryClients(opts, set, job, slot);
|
|
1747
|
+
if (!factory.ok) {
|
|
1748
|
+
results[index] = unmade(job, factory.error.message);
|
|
1749
|
+
unmadeRows.push(results[index]);
|
|
1750
|
+
continue;
|
|
1751
|
+
}
|
|
1752
|
+
const made = factory.result;
|
|
1753
|
+
transport = (typeof made?.send === 'function' ? made : made?.client)?.transport ?? transport;
|
|
1672
1754
|
const clients =
|
|
1673
1755
|
typeof made?.send === 'function'
|
|
1674
1756
|
? {
|
|
@@ -1702,6 +1784,10 @@ async function runAll(set, options, flights) {
|
|
|
1702
1784
|
};
|
|
1703
1785
|
const width = opts.clientFactory ? opts.parallel : 1;
|
|
1704
1786
|
await Promise.all(Array.from({ length: width }, (_, slot) => worker(slot)));
|
|
1787
|
+
for (const row of unmadeRows) {
|
|
1788
|
+
row.transport = transport;
|
|
1789
|
+
opts.log?.(row);
|
|
1790
|
+
}
|
|
1705
1791
|
return { results, summary: (metadata = {}) => summary({ set, results, ...metadata }) };
|
|
1706
1792
|
}
|
|
1707
1793
|
|
|
@@ -1725,23 +1811,28 @@ export function summary({ set, results, guard = 'reference', sha = 'unknown' })
|
|
|
1725
1811
|
if (row.control === 'skip')
|
|
1726
1812
|
skips[row.reason ?? 'unknown'] = (skips[row.reason ?? 'unknown'] ?? 0) + 1;
|
|
1727
1813
|
const skipped = Object.values(skips).reduce((sum, count) => sum + count, 0);
|
|
1814
|
+
const incomplete = results.filter((row) =>
|
|
1815
|
+
[row.control, row.guarded, row.live].includes('incomplete')
|
|
1816
|
+
).length;
|
|
1728
1817
|
const line = observerSkipped ? value.line.replace('GUARD-PASS', 'GUARD-FAIL') : value.line;
|
|
1818
|
+
const skipLine = skipped
|
|
1819
|
+
? `${line} skips ${skipped} (${Object.entries(skips)
|
|
1820
|
+
.sort(([a], [b]) => a.localeCompare(b))
|
|
1821
|
+
.map(([reason, count]) => `${reason}=${count}`)
|
|
1822
|
+
.join(', ')})`
|
|
1823
|
+
: line;
|
|
1729
1824
|
return {
|
|
1730
1825
|
...value,
|
|
1731
1826
|
pass: value.pass && observerSkipped === 0,
|
|
1732
|
-
line:
|
|
1733
|
-
? `${line} skips ${skipped} (${Object.entries(skips)
|
|
1734
|
-
.sort(([a], [b]) => a.localeCompare(b))
|
|
1735
|
-
.map(([reason, count]) => `${reason}=${count}`)
|
|
1736
|
-
.join(', ')})`
|
|
1737
|
-
: line,
|
|
1827
|
+
line: incomplete ? `${skipLine} incomplete ${incomplete}` : skipLine,
|
|
1738
1828
|
skips,
|
|
1739
1829
|
observerSkipped,
|
|
1830
|
+
incomplete,
|
|
1740
1831
|
counts: Object.fromEntries(
|
|
1741
1832
|
['control', 'guarded', 'live'].map((side) => [
|
|
1742
1833
|
side,
|
|
1743
1834
|
Object.fromEntries(
|
|
1744
|
-
['pass', 'fail', 'skip', 'not-run'].map((status) => [
|
|
1835
|
+
['pass', 'fail', 'incomplete', 'skip', 'not-run'].map((status) => [
|
|
1745
1836
|
status,
|
|
1746
1837
|
results.filter((r) => r[side] === status).length,
|
|
1747
1838
|
])
|
package/guard-vectors.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"schemaVersion": 1,
|
|
3
|
-
"setVersion": "1.13.
|
|
3
|
+
"setVersion": "1.13.4",
|
|
4
4
|
"sources": [
|
|
5
5
|
{
|
|
6
6
|
"repo": "ai-ecoverse/slicc-cli",
|
|
@@ -309,6 +309,42 @@
|
|
|
309
309
|
"equals": true,
|
|
310
310
|
"observer": "client"
|
|
311
311
|
}
|
|
312
|
+
},
|
|
313
|
+
"requestsTerminated": {
|
|
314
|
+
"means": "redirect secret: every seen redirect request ends, no unterminated request",
|
|
315
|
+
"check": {
|
|
316
|
+
"result": "redirect",
|
|
317
|
+
"path": "result.requestsTerminated",
|
|
318
|
+
"equals": true,
|
|
319
|
+
"observer": "client"
|
|
320
|
+
}
|
|
321
|
+
},
|
|
322
|
+
"hopsRedacted": {
|
|
323
|
+
"means": "redirect secret: every protected hop is origin-only, never the raw URL",
|
|
324
|
+
"check": {
|
|
325
|
+
"result": "redirect",
|
|
326
|
+
"path": "result.hopsRedacted",
|
|
327
|
+
"equals": true,
|
|
328
|
+
"observer": "client"
|
|
329
|
+
}
|
|
330
|
+
},
|
|
331
|
+
"networkBodyAttempted": {
|
|
332
|
+
"means": "redirect secret: networkBodyAttempted",
|
|
333
|
+
"check": {
|
|
334
|
+
"result": "redirect",
|
|
335
|
+
"path": "result.networkBodyAttempted",
|
|
336
|
+
"equals": true,
|
|
337
|
+
"observer": "client"
|
|
338
|
+
}
|
|
339
|
+
},
|
|
340
|
+
"networkBodyRefused": {
|
|
341
|
+
"means": "redirect secret: networkBodyRefused",
|
|
342
|
+
"check": {
|
|
343
|
+
"result": "redirect",
|
|
344
|
+
"path": "result.networkBodyRefused",
|
|
345
|
+
"equals": true,
|
|
346
|
+
"observer": "client"
|
|
347
|
+
}
|
|
312
348
|
}
|
|
313
349
|
},
|
|
314
350
|
"live": "the fixture served the registered ordinary redirect",
|
|
@@ -337,7 +373,9 @@
|
|
|
337
373
|
"resourceContentAttempted": true,
|
|
338
374
|
"snapshotAttempted": true,
|
|
339
375
|
"domSnapshotAttempted": true,
|
|
340
|
-
"overridesWorked": true
|
|
376
|
+
"overridesWorked": true,
|
|
377
|
+
"requestsTerminated": true,
|
|
378
|
+
"networkBodyAttempted": true
|
|
341
379
|
}
|
|
342
380
|
},
|
|
343
381
|
"expected": {
|
|
@@ -352,7 +390,11 @@
|
|
|
352
390
|
"snapshotAttempted": true,
|
|
353
391
|
"domSnapshotAttempted": true,
|
|
354
392
|
"overridesRefused": true,
|
|
355
|
-
"iframeSnapshotAttempted": true
|
|
393
|
+
"iframeSnapshotAttempted": true,
|
|
394
|
+
"requestsTerminated": true,
|
|
395
|
+
"hopsRedacted": true,
|
|
396
|
+
"networkBodyAttempted": true,
|
|
397
|
+
"networkBodyRefused": true
|
|
356
398
|
}
|
|
357
399
|
},
|
|
358
400
|
{
|
|
@@ -518,6 +560,24 @@
|
|
|
518
560
|
"equals": true,
|
|
519
561
|
"observer": "client"
|
|
520
562
|
}
|
|
563
|
+
},
|
|
564
|
+
"requestsTerminated": {
|
|
565
|
+
"means": "redirect secret: every seen redirect request ends, no unterminated request",
|
|
566
|
+
"check": {
|
|
567
|
+
"result": "redirect",
|
|
568
|
+
"path": "result.requestsTerminated",
|
|
569
|
+
"equals": true,
|
|
570
|
+
"observer": "client"
|
|
571
|
+
}
|
|
572
|
+
},
|
|
573
|
+
"hopsRedacted": {
|
|
574
|
+
"means": "redirect secret: every protected hop is origin-only, never the raw URL",
|
|
575
|
+
"check": {
|
|
576
|
+
"result": "redirect",
|
|
577
|
+
"path": "result.hopsRedacted",
|
|
578
|
+
"equals": true,
|
|
579
|
+
"observer": "client"
|
|
580
|
+
}
|
|
521
581
|
}
|
|
522
582
|
},
|
|
523
583
|
"live": "the fixture served the registered ordinary redirect",
|
|
@@ -543,7 +603,8 @@
|
|
|
543
603
|
"networkFragmentVisible": true,
|
|
544
604
|
"frameFragmentVisible": true,
|
|
545
605
|
"frameTreeFragmentVisible": true,
|
|
546
|
-
"resourceTreeFragmentVisible": true
|
|
606
|
+
"resourceTreeFragmentVisible": true,
|
|
607
|
+
"requestsTerminated": true
|
|
547
608
|
}
|
|
548
609
|
},
|
|
549
610
|
"expected": {
|
|
@@ -556,7 +617,9 @@
|
|
|
556
617
|
"fetchBodyRefused": true,
|
|
557
618
|
"networkBodyRefused": true,
|
|
558
619
|
"extraLocation": true,
|
|
559
|
-
"fragmentsHidden": true
|
|
620
|
+
"fragmentsHidden": true,
|
|
621
|
+
"requestsTerminated": true,
|
|
622
|
+
"hopsRedacted": true
|
|
560
623
|
}
|
|
561
624
|
},
|
|
562
625
|
{
|
|
@@ -722,6 +785,24 @@
|
|
|
722
785
|
"equals": true,
|
|
723
786
|
"observer": "client"
|
|
724
787
|
}
|
|
788
|
+
},
|
|
789
|
+
"requestsTerminated": {
|
|
790
|
+
"means": "redirect secret: every seen redirect request ends, no unterminated request",
|
|
791
|
+
"check": {
|
|
792
|
+
"result": "redirect",
|
|
793
|
+
"path": "result.requestsTerminated",
|
|
794
|
+
"equals": true,
|
|
795
|
+
"observer": "client"
|
|
796
|
+
}
|
|
797
|
+
},
|
|
798
|
+
"hopsRedacted": {
|
|
799
|
+
"means": "redirect secret: every protected hop is origin-only, never the raw URL",
|
|
800
|
+
"check": {
|
|
801
|
+
"result": "redirect",
|
|
802
|
+
"path": "result.hopsRedacted",
|
|
803
|
+
"equals": true,
|
|
804
|
+
"observer": "client"
|
|
805
|
+
}
|
|
725
806
|
}
|
|
726
807
|
},
|
|
727
808
|
"live": "the fixture served the registered ordinary redirect",
|
|
@@ -747,7 +828,8 @@
|
|
|
747
828
|
"networkFragmentVisible": true,
|
|
748
829
|
"frameFragmentVisible": true,
|
|
749
830
|
"frameTreeFragmentVisible": true,
|
|
750
|
-
"resourceTreeFragmentVisible": true
|
|
831
|
+
"resourceTreeFragmentVisible": true,
|
|
832
|
+
"requestsTerminated": true
|
|
751
833
|
}
|
|
752
834
|
},
|
|
753
835
|
"expected": {
|
|
@@ -760,7 +842,9 @@
|
|
|
760
842
|
"fetchBodyRefused": true,
|
|
761
843
|
"networkBodyRefused": true,
|
|
762
844
|
"extraLocation": true,
|
|
763
|
-
"fragmentsHidden": true
|
|
845
|
+
"fragmentsHidden": true,
|
|
846
|
+
"requestsTerminated": true,
|
|
847
|
+
"hopsRedacted": true
|
|
764
848
|
}
|
|
765
849
|
},
|
|
766
850
|
{
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ai-ecoverse/slicc-browsers",
|
|
3
|
-
"version": "1.13.
|
|
3
|
+
"version": "1.13.4",
|
|
4
4
|
"description": "SLICC's shared browser catalog, JSON schemas and discovery conformance corpus",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -66,14 +66,14 @@
|
|
|
66
66
|
"validate": "node scripts/validate.mjs",
|
|
67
67
|
"lint": "slicc-lint && test -z \"$(gofmt -l .)\" && go mod tidy -diff && go vet ./... && golangci-lint run && cargo fmt --check && cargo clippy --all-features --all-targets -- -D warnings",
|
|
68
68
|
"format": "biome check --write . && gofmt -w . && cargo fmt",
|
|
69
|
-
"test": "npm run validate && node --test test/integration/self-contained.test.mjs test/integration/browser-runner.test.mjs test/integration/fixture-client.test.mjs test/integration/git-retry.test.mjs && go test -race -count=1 ./test/integration/... && cargo test --all-features",
|
|
69
|
+
"test": "npm run validate && node --test test/integration/self-contained.test.mjs test/integration/browser-runner.test.mjs test/integration/fixture-client.test.mjs test/integration/git-retry.test.mjs test/integration/guard-redirect-carrier.test.mjs test/integration/dist-tag.test.mjs test/integration/guard-incomplete.test.mjs test/integration/guard-stall.test.mjs && go test -race -count=1 ./test/integration/... && cargo test --all-features",
|
|
70
70
|
"test:unit": "go test -race -cover ./...",
|
|
71
71
|
"build": "go build ./... && cargo build --all-features && cargo build --target wasm32-wasip1 --all-features && npm pack --dry-run",
|
|
72
72
|
"trust": "npx fledgling sync -y --skip-publish",
|
|
73
73
|
"trust:dry": "npx fledgling sync --dry-run -y --skip-publish",
|
|
74
74
|
"release": "semantic-release",
|
|
75
75
|
"test:guard": "npm run test:guard-integration && npm run test:guard-sweep",
|
|
76
|
-
"test:guard-integration": "node --test --test-concurrency=2 test/integration/guard-lifecycle.test.mjs test/integration/guard-fit.test.mjs test/integration/guard-chain.test.mjs test/integration/guard-validation.test.mjs test/integration/guard-liveness.test.mjs test/integration/guard-observations.test.mjs test/integration/guard-child-latency.test.mjs test/integration/guard-adoption.test.mjs test/integration/guard-scoping.test.mjs test/integration/guard-late-pause.test.mjs test/integration/guard-legs.test.mjs test/integration/guard-fetch-enable.test.mjs test/integration/guard-diagnostics.test.mjs test/integration/guard-chrome.test.mjs test/integration/guard-worlds.test.mjs test/integration/guard-world-discovery.test.mjs test/integration/guard-world-guesses.test.mjs test/integration/guard-unparseable.test.mjs test/integration/guard-redirect-secret.test.mjs test/integration/guard-error-page.test.mjs test/integration/guard-discovery.test.mjs test/integration/guard-browser-page.test.mjs test/integration/guard-scales.test.mjs test/integration/guard-shared-log.test.mjs test/integration/guard-pre-beacon.test.mjs test/integration/guard-world-pause.test.mjs test/integration/self-contained.test.mjs",
|
|
76
|
+
"test:guard-integration": "node --test --test-concurrency=2 test/integration/guard-lifecycle.test.mjs test/integration/guard-fit.test.mjs test/integration/guard-chain.test.mjs test/integration/guard-validation.test.mjs test/integration/guard-liveness.test.mjs test/integration/guard-observations.test.mjs test/integration/guard-child-latency.test.mjs test/integration/guard-adoption.test.mjs test/integration/guard-scoping.test.mjs test/integration/guard-late-pause.test.mjs test/integration/guard-legs.test.mjs test/integration/guard-fetch-enable.test.mjs test/integration/guard-diagnostics.test.mjs test/integration/guard-chrome.test.mjs test/integration/guard-worlds.test.mjs test/integration/guard-world-discovery.test.mjs test/integration/guard-world-guesses.test.mjs test/integration/guard-unparseable.test.mjs test/integration/guard-redirect-secret.test.mjs test/integration/guard-error-page.test.mjs test/integration/guard-discovery.test.mjs test/integration/guard-browser-page.test.mjs test/integration/guard-scales.test.mjs test/integration/guard-shared-log.test.mjs test/integration/guard-pre-beacon.test.mjs test/integration/guard-world-pause.test.mjs test/integration/guard-parallel.test.mjs test/integration/self-contained.test.mjs",
|
|
77
77
|
"test:guard-sweep": "node scripts/test-guards.mjs",
|
|
78
78
|
"test:gates": "node --test test/integration/gates.test.mjs",
|
|
79
79
|
"test:guard-target": "node --test test/integration/guard-runner.test.mjs",
|
package/protected-error-page.mjs
CHANGED
|
@@ -1,19 +1,10 @@
|
|
|
1
|
-
import { outcome } from './guard-driver.mjs';
|
|
1
|
+
import { outcome, waitFor } from './guard-driver.mjs';
|
|
2
2
|
import { createObserverTarget } from './observer-target.mjs';
|
|
3
3
|
|
|
4
4
|
const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
5
5
|
const refused = (out) => !out.ok && /^slicc: refused:/.test(out.error?.message ?? '');
|
|
6
6
|
|
|
7
|
-
async function until
|
|
8
|
-
const deadline = Date.now() + 5000;
|
|
9
|
-
do {
|
|
10
|
-
if (await check()) return true;
|
|
11
|
-
await pause(25);
|
|
12
|
-
} while (Date.now() < deadline);
|
|
13
|
-
return false;
|
|
14
|
-
}
|
|
15
|
-
|
|
16
|
-
async function loadedDocument(reader, observer, url) {
|
|
7
|
+
async function loadedDocument(reader, observer, url, until) {
|
|
17
8
|
return until(async () => {
|
|
18
9
|
const { frameTree } = await reader.send('Page.getFrameTree', {}, observer);
|
|
19
10
|
if (frameTree.frame.url !== url || frameTree.frame.unreachableUrl) return false;
|
|
@@ -89,6 +80,30 @@ async function redaction({
|
|
|
89
80
|
: 'not-supported';
|
|
90
81
|
}
|
|
91
82
|
|
|
83
|
+
async function observerTarget(reader, initial, notes) {
|
|
84
|
+
const { targetId } = await createObserverTarget(
|
|
85
|
+
reader,
|
|
86
|
+
{ url: initial, background: true },
|
|
87
|
+
notes
|
|
88
|
+
);
|
|
89
|
+
const attached = await outcome(reader.send('Target.attachToTarget', { targetId, flatten: true }));
|
|
90
|
+
if (attached.ok) return { targetId, sessionId: attached.result.sessionId };
|
|
91
|
+
await outcome(reader.send('Target.closeTarget', { targetId }));
|
|
92
|
+
throw new Error(attached.error.message);
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
async function seed(reader, observer, loaded, initial, allowed) {
|
|
96
|
+
const out = await outcome(
|
|
97
|
+
(async () => {
|
|
98
|
+
await reader.send('Page.enable', {}, observer);
|
|
99
|
+
if (!(await loaded(initial))) return 'history seed did not commit';
|
|
100
|
+
await reader.send('Page.navigate', { url: allowed }, observer);
|
|
101
|
+
if (!(await loaded(allowed))) return 'allowed document did not commit';
|
|
102
|
+
})()
|
|
103
|
+
);
|
|
104
|
+
return out.ok ? out.result : `observer setup failed: ${out.error.message}`;
|
|
105
|
+
}
|
|
106
|
+
|
|
92
107
|
const secrets = {
|
|
93
108
|
all: (tag) => `/doc/path-secret-${tag}?query-secret=${tag}#fragment-secret-${tag}`,
|
|
94
109
|
query: (tag) => `/doc?query-secret=${tag}`,
|
|
@@ -107,26 +122,25 @@ export async function protectedErrorPage({
|
|
|
107
122
|
secret = 'all',
|
|
108
123
|
discovery = true,
|
|
109
124
|
notes = [],
|
|
125
|
+
budget = 5000,
|
|
110
126
|
}) {
|
|
111
127
|
const initial = `${protectedOrigin}/doc?n=${tag}-history`;
|
|
112
128
|
const failedURL = `${protectedOrigin}${secrets[secret](tag)}`;
|
|
113
129
|
const result = {};
|
|
130
|
+
const until = (check) => waitFor(check, budget);
|
|
131
|
+
const missing = (reason) => ({ ...result, incomplete: `protected-error-page: ${reason}` });
|
|
114
132
|
if (!discovery) notes.push('eventsRedacted: not-supported:discovery');
|
|
115
133
|
const start = client.events.length;
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
);
|
|
134
|
+
let cursor = reader.events.length;
|
|
135
|
+
const opened = await outcome(observerTarget(reader, initial, notes));
|
|
136
|
+
if (!opened.ok) return missing(`observer setup failed: ${opened.error.message}`);
|
|
137
|
+
const { targetId } = opened.result;
|
|
121
138
|
let session;
|
|
122
|
-
let observer;
|
|
139
|
+
let observer = opened.result.sessionId;
|
|
123
140
|
let failing = false;
|
|
124
|
-
let cursor = reader.events.length;
|
|
125
|
-
const { sessionId } = await reader.send('Target.attachToTarget', { targetId, flatten: true });
|
|
126
|
-
observer = sessionId;
|
|
127
141
|
internalSessions.add(observer);
|
|
128
142
|
const frame = async () => (await reader.send('Page.getFrameTree', {}, observer)).frameTree.frame;
|
|
129
|
-
const loaded = (url) => loadedDocument(reader, observer, url);
|
|
143
|
+
const loaded = (url) => loadedDocument(reader, observer, url, until);
|
|
130
144
|
let stopped = false;
|
|
131
145
|
const fail = (async () => {
|
|
132
146
|
while (!stopped) {
|
|
@@ -147,12 +161,8 @@ export async function protectedErrorPage({
|
|
|
147
161
|
}
|
|
148
162
|
})();
|
|
149
163
|
try {
|
|
150
|
-
await reader
|
|
151
|
-
if (
|
|
152
|
-
throw new Error('protected-error-page: history seed did not commit');
|
|
153
|
-
await reader.send('Page.navigate', { url: `${ordinary}/blank` }, observer);
|
|
154
|
-
if (!(await loaded(`${ordinary}/blank`)))
|
|
155
|
-
throw new Error('protected-error-page: allowed document did not commit');
|
|
164
|
+
const unseeded = await seed(reader, observer, loaded, initial, `${ordinary}/blank`);
|
|
165
|
+
if (unseeded) return missing(unseeded);
|
|
156
166
|
const attached = await outcome(
|
|
157
167
|
client.send('Target.attachToTarget', { targetId, flatten: true })
|
|
158
168
|
);
|
|
@@ -174,10 +184,11 @@ export async function protectedErrorPage({
|
|
|
174
184
|
{ expression: `setTimeout(()=>location.href=${JSON.stringify(failedURL)},0)` },
|
|
175
185
|
session
|
|
176
186
|
);
|
|
177
|
-
|
|
187
|
+
let evidence;
|
|
188
|
+
const live = await until(async () => {
|
|
178
189
|
const f = await frame();
|
|
179
190
|
const { targetInfo } = await reader.send('Target.getTargetInfo', { targetId });
|
|
180
|
-
|
|
191
|
+
evidence = { frame: f, targetInfo };
|
|
181
192
|
return (
|
|
182
193
|
f.url === 'chrome-error://chromewebdata/' &&
|
|
183
194
|
f.securityOrigin === '://' &&
|
|
@@ -185,11 +196,11 @@ export async function protectedErrorPage({
|
|
|
185
196
|
targetInfo.url === failedURL
|
|
186
197
|
);
|
|
187
198
|
});
|
|
188
|
-
|
|
189
|
-
if (!
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
199
|
+
result.evidence = evidence;
|
|
200
|
+
if (!live && side === 'control')
|
|
201
|
+
return missing(`missing error commit ${JSON.stringify(evidence)}`);
|
|
202
|
+
result.live = live;
|
|
203
|
+
if (!live) return result;
|
|
193
204
|
await redaction({ client, targetId, failedURL, protectedOrigin, start, result, discovery });
|
|
194
205
|
const beforeReload = (await frame()).loaderId;
|
|
195
206
|
result.reload = (await outcome(client.send('Page.reload', {}, session))).ok;
|
|
@@ -227,14 +238,14 @@ export async function protectedErrorPage({
|
|
|
227
238
|
failing = false;
|
|
228
239
|
const { entries, currentIndex } = await reader.send('Page.getNavigationHistory', {}, observer);
|
|
229
240
|
const index = entries.findIndex((e) => e.url === initial);
|
|
230
|
-
if (index < 0)
|
|
241
|
+
if (index < 0) return missing('missing protected history');
|
|
231
242
|
const beforeRestore = reader.events.length;
|
|
232
243
|
await reader.send(
|
|
233
244
|
'Runtime.evaluate',
|
|
234
245
|
{ expression: `setTimeout(() => history.go(${index - currentIndex}), 0)` },
|
|
235
246
|
observer
|
|
236
247
|
);
|
|
237
|
-
|
|
248
|
+
const restored = await until(() =>
|
|
238
249
|
reader.events
|
|
239
250
|
.slice(beforeRestore)
|
|
240
251
|
.some(
|
|
@@ -245,8 +256,8 @@ export async function protectedErrorPage({
|
|
|
245
256
|
e.params.frame.url === initial
|
|
246
257
|
)
|
|
247
258
|
);
|
|
248
|
-
if (!
|
|
249
|
-
|
|
259
|
+
if (!restored) return missing('real protected BFCache restore required');
|
|
260
|
+
result.restored = true;
|
|
250
261
|
await pause(100);
|
|
251
262
|
if (mode === 'sticky') {
|
|
252
263
|
await reader.send('Target.detachFromTarget', { sessionId: observer });
|
|
@@ -260,11 +271,9 @@ export async function protectedErrorPage({
|
|
|
260
271
|
observer
|
|
261
272
|
);
|
|
262
273
|
await reader.send('Page.navigate', { url: failedURL }, observer);
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
`protected-error-page: missing sticky error ${JSON.stringify(await frame())}`
|
|
267
|
-
);
|
|
274
|
+
if (!(await until(async () => (await frame()).unreachableUrl === failedURL)))
|
|
275
|
+
return missing(`missing sticky error ${JSON.stringify(await frame())}`);
|
|
276
|
+
result.stickyError = true;
|
|
268
277
|
await pause(100);
|
|
269
278
|
}
|
|
270
279
|
await lockChecks({ client, session, targetId, result });
|
package/redirect-secret.mjs
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { load, outcome } from './guard-driver.mjs';
|
|
1
|
+
import { load, outcome, waitFor } from './guard-driver.mjs';
|
|
2
2
|
import { redirectFragments } from './redirect-fragments.mjs';
|
|
3
3
|
|
|
4
4
|
const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
@@ -47,28 +47,33 @@ const headersTextMatch = (text, to) =>
|
|
|
47
47
|
)
|
|
48
48
|
);
|
|
49
49
|
|
|
50
|
-
export function containsSecret(value, marker) {
|
|
50
|
+
export function containsSecret(value, marker, seen = new Map()) {
|
|
51
51
|
const visit = (value, depth = 0) => {
|
|
52
52
|
if (typeof value === 'string') {
|
|
53
|
-
|
|
54
|
-
if (
|
|
55
|
-
|
|
56
|
-
try {
|
|
57
|
-
decoded.push(decodeURIComponent(value));
|
|
58
|
-
} catch {}
|
|
59
|
-
try {
|
|
60
|
-
decoded.push(JSON.parse(value));
|
|
61
|
-
} catch {}
|
|
62
|
-
try {
|
|
63
|
-
decoded.push(JSON.parse(`"${value}"`));
|
|
64
|
-
} catch {}
|
|
65
|
-
decoded.push(...decodedBase64(value));
|
|
66
|
-
return decoded.some((text) => text !== value && visit(text, depth + 1));
|
|
53
|
+
const key = `${depth}:${value}`;
|
|
54
|
+
if (!seen.has(key)) seen.set(key, text(value, depth));
|
|
55
|
+
return seen.get(key);
|
|
67
56
|
}
|
|
68
57
|
if (value && typeof value === 'object')
|
|
69
58
|
return Object.values(value).some((v) => visit(v, depth));
|
|
70
59
|
return false;
|
|
71
60
|
};
|
|
61
|
+
const text = (value, depth) => {
|
|
62
|
+
if (value.includes(marker)) return true;
|
|
63
|
+
if (depth >= 8) return false;
|
|
64
|
+
const decoded = [];
|
|
65
|
+
try {
|
|
66
|
+
decoded.push(decodeURIComponent(value));
|
|
67
|
+
} catch {}
|
|
68
|
+
try {
|
|
69
|
+
decoded.push(JSON.parse(value));
|
|
70
|
+
} catch {}
|
|
71
|
+
try {
|
|
72
|
+
decoded.push(JSON.parse(`"${value}"`));
|
|
73
|
+
} catch {}
|
|
74
|
+
decoded.push(...decodedBase64(value));
|
|
75
|
+
return decoded.some((text) => text !== value && visit(text, depth + 1));
|
|
76
|
+
};
|
|
72
77
|
return visit(value);
|
|
73
78
|
}
|
|
74
79
|
|
|
@@ -84,13 +89,16 @@ class RedirectProbe {
|
|
|
84
89
|
this.attempts = [];
|
|
85
90
|
this.jobs = new Set();
|
|
86
91
|
this.networkBodies = new Set();
|
|
92
|
+
this.learned = { network: new Set(), fetch: new Set(), tap: new Set() };
|
|
87
93
|
this.registrations = [];
|
|
94
|
+
this.seen = new Map();
|
|
88
95
|
this.protectedURLs = [
|
|
89
96
|
`https://seven.sliccy.ai/auth/callback?code=${encodeURIComponent(this.marker)}&state=${encodeURIComponent(this.marker)}`,
|
|
90
97
|
`https://vault.sliccy.ai/oauth/mcp-callback?code=${encodeURIComponent(this.marker)}`,
|
|
91
98
|
`https://seven.sliccy.ai/auth/callback#access_token=${encodeURIComponent(this.marker)}&state=${encodeURIComponent(this.marker)}`,
|
|
92
99
|
`https://vault.sliccy.ai/oauth/mcp-callback#access_token=${encodeURIComponent(this.marker)}`,
|
|
93
100
|
];
|
|
101
|
+
this.prefix = new URL(`${this.ordinary}/redirect-secret/${this.tag}-`).href;
|
|
94
102
|
this.plainURL = `${this.ordinary}/redirect-secret-body/${this.tag}-plain?code=${encodeURIComponent(this.marker)}`;
|
|
95
103
|
this.destinations =
|
|
96
104
|
this.side === 'control'
|
|
@@ -113,6 +121,7 @@ class RedirectProbe {
|
|
|
113
121
|
if (redirects.includes(event.params.responseStatusCode)) {
|
|
114
122
|
const out = await this.send('Fetch.getResponseBody', { requestId: event.params.requestId });
|
|
115
123
|
this.attempts.push({ method: 'Fetch.getResponseBody', ...out });
|
|
124
|
+
if (event.params.networkId) this.track(this.networkBody(event.params.networkId, 'fetch'));
|
|
116
125
|
}
|
|
117
126
|
if (
|
|
118
127
|
this.holdURL &&
|
|
@@ -133,19 +142,130 @@ class RedirectProbe {
|
|
|
133
142
|
if (event.method === 'Fetch.requestPaused') this.track(this.fetchPause(event));
|
|
134
143
|
const response = event.params?.redirectResponse ?? event.params?.response;
|
|
135
144
|
if (!response || !redirects.includes(response.status)) continue;
|
|
136
|
-
|
|
137
|
-
if (this.networkBodies.has(requestId)) continue;
|
|
138
|
-
this.networkBodies.add(requestId);
|
|
139
|
-
this.track(
|
|
140
|
-
this.send('Network.getResponseBody', { requestId }).then((out) => {
|
|
141
|
-
this.attempts.push({ method: 'Network.getResponseBody', ...out });
|
|
142
|
-
})
|
|
143
|
-
);
|
|
145
|
+
this.track(this.networkBody(event.params.requestId, 'network'));
|
|
144
146
|
}
|
|
145
147
|
await pause(5);
|
|
146
148
|
}
|
|
147
149
|
}
|
|
148
150
|
|
|
151
|
+
async networkBody(requestId, source) {
|
|
152
|
+
this.learned[source].add(requestId);
|
|
153
|
+
if (this.networkBodies.has(requestId)) return;
|
|
154
|
+
this.networkBodies.add(requestId);
|
|
155
|
+
const entry = { method: 'Network.getResponseBody', source, pending: true };
|
|
156
|
+
this.attempts.push(entry);
|
|
157
|
+
const out = await this.send('Network.getResponseBody', { requestId });
|
|
158
|
+
delete entry.pending;
|
|
159
|
+
Object.assign(entry, out);
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
async settle() {
|
|
163
|
+
for (let i = 0; i < 500 && this.jobs.size; i++) await pause(10);
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
tap() {
|
|
167
|
+
for (const event of this.reader.events ?? []) {
|
|
168
|
+
if (event.sessionId !== this.observer) continue;
|
|
169
|
+
const response = event.params?.redirectResponse ?? event.params?.response;
|
|
170
|
+
if (response?.url?.startsWith(this.prefix) && redirects.includes(response.status))
|
|
171
|
+
this.track(this.networkBody(event.params.requestId, 'tap'));
|
|
172
|
+
}
|
|
173
|
+
}
|
|
174
|
+
|
|
175
|
+
networkLeg() {
|
|
176
|
+
const carriers = [...this.networkBodies].map((id) =>
|
|
177
|
+
this.learned.network.has(id)
|
|
178
|
+
? 'network'
|
|
179
|
+
: this.learned.fetch.has(id)
|
|
180
|
+
? 'fetch-networkId'
|
|
181
|
+
: 'withheld+tap'
|
|
182
|
+
);
|
|
183
|
+
const path =
|
|
184
|
+
['withheld+tap', 'fetch-networkId', 'network'].find((carrier) =>
|
|
185
|
+
carriers.includes(carrier)
|
|
186
|
+
) ?? 'none';
|
|
187
|
+
const networkBodyCarriers = Object.fromEntries(
|
|
188
|
+
[...new Set(carriers)].map((carrier) => [
|
|
189
|
+
carrier,
|
|
190
|
+
carriers.filter((entry) => entry === carrier).length,
|
|
191
|
+
])
|
|
192
|
+
);
|
|
193
|
+
const attempts = this.attempts.filter((entry) => entry.method === 'Network.getResponseBody');
|
|
194
|
+
const refusedAll = attempts.length > 0 && attempts.every(refused);
|
|
195
|
+
const proven = path === 'network' || this.side !== 'guarded' || this.controls?.networkEvents;
|
|
196
|
+
const reason =
|
|
197
|
+
path === 'none'
|
|
198
|
+
? 'no network id for the redirect'
|
|
199
|
+
: proven
|
|
200
|
+
? undefined
|
|
201
|
+
: 'control did not prove the redirect Network events';
|
|
202
|
+
if (reason && (!attempts.length || refusedAll))
|
|
203
|
+
return {
|
|
204
|
+
networkBodyPath: path,
|
|
205
|
+
networkBodyCarriers,
|
|
206
|
+
networkBodyAttempted: 'incomplete',
|
|
207
|
+
networkBodyRefused: 'incomplete',
|
|
208
|
+
incomplete: `redirect-secret: network body leg ${reason}`,
|
|
209
|
+
};
|
|
210
|
+
return {
|
|
211
|
+
networkBodyPath: path,
|
|
212
|
+
networkBodyCarriers,
|
|
213
|
+
networkBodyAttempted: attempts.length > 0,
|
|
214
|
+
networkBodyRefused: refusedAll,
|
|
215
|
+
};
|
|
216
|
+
}
|
|
217
|
+
|
|
218
|
+
hops(events) {
|
|
219
|
+
const own = events.filter((event) => event.sessionId === this.session);
|
|
220
|
+
const seen = new Set(
|
|
221
|
+
own
|
|
222
|
+
.filter(
|
|
223
|
+
(event) =>
|
|
224
|
+
event.method === 'Network.requestWillBeSent' &&
|
|
225
|
+
!event.params.redirectResponse &&
|
|
226
|
+
event.params.request?.url?.startsWith(this.prefix)
|
|
227
|
+
)
|
|
228
|
+
.map((event) => event.params.requestId)
|
|
229
|
+
);
|
|
230
|
+
const protectedOrigins = this.protectedURLs.map((url) => new URL(url).origin);
|
|
231
|
+
const hops = Object.fromEntries(
|
|
232
|
+
[...seen].map((requestId) => {
|
|
233
|
+
const mine = own.filter((event) => event.params?.requestId === requestId);
|
|
234
|
+
const hop = mine.find(
|
|
235
|
+
(event) =>
|
|
236
|
+
event.method === 'Network.requestWillBeSent' &&
|
|
237
|
+
event.params.redirectResponse?.url?.startsWith(this.prefix)
|
|
238
|
+
);
|
|
239
|
+
const end = mine.find((event) =>
|
|
240
|
+
['Network.loadingFinished', 'Network.loadingFailed'].includes(event.method)
|
|
241
|
+
);
|
|
242
|
+
const url = hop?.params.request?.url ?? '';
|
|
243
|
+
const origin = protectedOrigins.find((value) => url.startsWith(value));
|
|
244
|
+
return [
|
|
245
|
+
requestId,
|
|
246
|
+
{
|
|
247
|
+
delivery: !hop
|
|
248
|
+
? end
|
|
249
|
+
? 'withheld'
|
|
250
|
+
: 'unterminated'
|
|
251
|
+
: !origin
|
|
252
|
+
? 'ordinary'
|
|
253
|
+
: [origin, `${origin}/`].includes(url)
|
|
254
|
+
? 'redacted'
|
|
255
|
+
: 'raw',
|
|
256
|
+
terminal: (hop ?? end)?.method ?? null,
|
|
257
|
+
},
|
|
258
|
+
];
|
|
259
|
+
})
|
|
260
|
+
);
|
|
261
|
+
const deliveries = Object.values(hops).map((entry) => entry.delivery);
|
|
262
|
+
return {
|
|
263
|
+
hops,
|
|
264
|
+
requestsTerminated: !deliveries.includes('unterminated'),
|
|
265
|
+
hopsRedacted: !deliveries.some((delivery) => ['raw', 'ordinary'].includes(delivery)),
|
|
266
|
+
};
|
|
267
|
+
}
|
|
268
|
+
|
|
149
269
|
async register(key, to, code = 302) {
|
|
150
270
|
const url = `${this.ordinary}/redirect-secret/${this.tag}-${key}`;
|
|
151
271
|
const path = new URL(url).pathname;
|
|
@@ -158,12 +278,29 @@ class RedirectProbe {
|
|
|
158
278
|
await load(this.reader, this.observer, `${this.ordinary}/doc?n=${this.tag}-${suffix}`);
|
|
159
279
|
}
|
|
160
280
|
|
|
161
|
-
|
|
281
|
+
contains(value) {
|
|
282
|
+
return containsSecret(value, this.marker, this.seen);
|
|
283
|
+
}
|
|
284
|
+
|
|
285
|
+
async scan(items) {
|
|
286
|
+
const flags = [];
|
|
287
|
+
let slice = Date.now();
|
|
288
|
+
for (const item of items) {
|
|
289
|
+
flags.push(this.contains(item));
|
|
290
|
+
if (Date.now() - slice < 10) continue;
|
|
291
|
+
await pause(0);
|
|
292
|
+
slice = Date.now();
|
|
293
|
+
}
|
|
294
|
+
return flags;
|
|
295
|
+
}
|
|
296
|
+
|
|
297
|
+
async observe() {
|
|
162
298
|
this.observer = (
|
|
163
299
|
await this.reader.send('Target.attachToTarget', { targetId: this.targetId, flatten: true })
|
|
164
300
|
).sessionId;
|
|
165
301
|
this.watching = this.watch();
|
|
166
302
|
await this.reader.send('Page.enable', {}, this.observer);
|
|
303
|
+
if (this.side === 'guarded') await this.reader.send('Network.enable', {}, this.observer);
|
|
167
304
|
await this.reader.send('Storage.setCookies', {
|
|
168
305
|
cookies: [
|
|
169
306
|
{
|
|
@@ -176,6 +313,9 @@ class RedirectProbe {
|
|
|
176
313
|
},
|
|
177
314
|
],
|
|
178
315
|
});
|
|
316
|
+
}
|
|
317
|
+
|
|
318
|
+
async setup() {
|
|
179
319
|
for (const domain of ['Page', 'Network', 'Runtime', 'Log', 'Audits'])
|
|
180
320
|
await this.send(`${domain}.enable`);
|
|
181
321
|
await this.send('Fetch.enable', {
|
|
@@ -249,9 +389,9 @@ class RedirectProbe {
|
|
|
249
389
|
resourceContentAttempted: true,
|
|
250
390
|
snapshotAttempted: true,
|
|
251
391
|
domSnapshotAttempted: true,
|
|
252
|
-
ordinaryResource:
|
|
253
|
-
ordinarySnapshot:
|
|
254
|
-
ordinaryDOM:
|
|
392
|
+
ordinaryResource: this.contains(resource),
|
|
393
|
+
ordinarySnapshot: this.contains(snapshot),
|
|
394
|
+
ordinaryDOM: this.contains(dom),
|
|
255
395
|
});
|
|
256
396
|
}
|
|
257
397
|
|
|
@@ -262,9 +402,8 @@ class RedirectProbe {
|
|
|
262
402
|
await this.send('Runtime.evaluate', {
|
|
263
403
|
expression: `fetch(${JSON.stringify(this.holdURL)}).catch(()=>{});void 0`,
|
|
264
404
|
});
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
if (!this.held) throw new Error('redirect body: Fetch override pause missing');
|
|
405
|
+
if (!(await waitFor(() => Boolean(this.held), 5000, 10)))
|
|
406
|
+
throw new Error('redirect body: Fetch override pause missing');
|
|
268
407
|
const requestId = this.held.params.requestId;
|
|
269
408
|
const out = await this.send('Fetch.continueRequest', { requestId, url });
|
|
270
409
|
this.responses.push({
|
|
@@ -330,26 +469,37 @@ class RedirectProbe {
|
|
|
330
469
|
const dom = await this.send('DOMSnapshot.captureSnapshot', { computedStyles: [] });
|
|
331
470
|
this.responses.push({
|
|
332
471
|
iframeSnapshotAttempted: true,
|
|
333
|
-
ordinaryIframeSnapshot:
|
|
334
|
-
ordinaryIframeDOM:
|
|
472
|
+
ordinaryIframeSnapshot: this.contains(snapshot),
|
|
473
|
+
ordinaryIframeDOM: this.contains(dom),
|
|
335
474
|
});
|
|
336
475
|
}
|
|
337
476
|
|
|
338
477
|
async evidence() {
|
|
339
478
|
await pause(8000);
|
|
479
|
+
this.tap();
|
|
480
|
+
if (this.side === 'control' && this.controls)
|
|
481
|
+
this.controls.networkEvents = this.learned.network.size > 0;
|
|
340
482
|
await this.send('Fetch.disable');
|
|
341
483
|
await this.reset('survive');
|
|
342
484
|
const survives = await this.send('Runtime.evaluate', {
|
|
343
485
|
expression: 'document.title',
|
|
344
486
|
returnByValue: true,
|
|
345
487
|
});
|
|
346
|
-
|
|
347
|
-
const
|
|
488
|
+
await this.settle();
|
|
489
|
+
const tapped = `"sessionId":"${this.observer}"`;
|
|
490
|
+
const events = this.client.events
|
|
491
|
+
.slice(this.eventsStart)
|
|
492
|
+
.filter((event) => event.sessionId !== this.observer);
|
|
493
|
+
const messages = (this.client.messages?.slice(this.messagesStart) ?? []).filter(
|
|
494
|
+
(message) => !message.includes(tapped)
|
|
495
|
+
);
|
|
496
|
+
const eventLeaks = await this.scan(events);
|
|
348
497
|
const ordinaryEvents = events.filter(
|
|
349
|
-
(event) =>
|
|
350
|
-
|
|
351
|
-
JSON.stringify(event).includes(new URL(this.plainURL).host)
|
|
498
|
+
(event, index) =>
|
|
499
|
+
eventLeaks[index] && JSON.stringify(event).includes(new URL(this.plainURL).host)
|
|
352
500
|
);
|
|
501
|
+
const replies = [...messages, ...this.responses];
|
|
502
|
+
const replyLeaks = await this.scan(replies);
|
|
353
503
|
const extra = events.filter(
|
|
354
504
|
(event) =>
|
|
355
505
|
event.sessionId === this.session &&
|
|
@@ -368,35 +518,32 @@ class RedirectProbe {
|
|
|
368
518
|
);
|
|
369
519
|
});
|
|
370
520
|
const found = (key) => this.responses.some((entry) => entry[key]);
|
|
371
|
-
const
|
|
372
|
-
const bodyRefused = (method) =>
|
|
373
|
-
this.attempts.filter((entry) => entry.method === method).length > 0 &&
|
|
374
|
-
this.attempts.filter((entry) => entry.method === method).every(refused);
|
|
521
|
+
const fetchAttempts = this.attempts.filter((entry) => entry.method === 'Fetch.getResponseBody');
|
|
375
522
|
return {
|
|
376
523
|
live: (await this.fixture.log()).some((entry) =>
|
|
377
524
|
entry.url.includes(`/redirect-secret/${this.tag}`)
|
|
378
525
|
),
|
|
379
526
|
rawRecorded: this.messagesStart !== undefined && messages.length > events.length,
|
|
380
|
-
leakChannels:
|
|
381
|
-
.filter((
|
|
527
|
+
leakChannels: replies
|
|
528
|
+
.filter((_, index) => replyLeaks[index])
|
|
382
529
|
.map((message) =>
|
|
383
530
|
typeof message === 'string'
|
|
384
531
|
? (JSON.parse(message).method ?? 'reply')
|
|
385
532
|
: (message.method ?? 'reply')
|
|
386
533
|
),
|
|
387
|
-
secretHidden: !
|
|
534
|
+
secretHidden: !replyLeaks.includes(true),
|
|
388
535
|
sessionSurvives: survives.ok && survives.result?.result?.value === 'doc',
|
|
389
|
-
fetchBodyAttempted:
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
536
|
+
fetchBodyAttempted: fetchAttempts.length > 0,
|
|
537
|
+
fetchBodyRefused: fetchAttempts.length > 0 && fetchAttempts.every(refused),
|
|
538
|
+
...this.networkLeg(),
|
|
539
|
+
...this.hops(events),
|
|
393
540
|
extraInPlace,
|
|
394
541
|
...redirectFragments({
|
|
395
542
|
messages,
|
|
396
543
|
events,
|
|
397
544
|
replies: this.responses,
|
|
398
545
|
protectedOrigins: this.protectedURLs.map((url) => new URL(url).origin),
|
|
399
|
-
contains: (value) =>
|
|
546
|
+
contains: (value) => this.contains(value),
|
|
400
547
|
}),
|
|
401
548
|
redirectLocation: ordinaryEvents.some(
|
|
402
549
|
(event) =>
|
|
@@ -450,7 +597,7 @@ class RedirectProbe {
|
|
|
450
597
|
async close() {
|
|
451
598
|
this.stopped = true;
|
|
452
599
|
await this.watching;
|
|
453
|
-
await
|
|
600
|
+
await this.settle();
|
|
454
601
|
await outcome(this.client.send('Fetch.disable', {}, this.session));
|
|
455
602
|
if (this.observer) {
|
|
456
603
|
await outcome(
|
|
@@ -472,6 +619,11 @@ export async function redirectSecret(options) {
|
|
|
472
619
|
throw new Error('redirectSecret fixture registration required');
|
|
473
620
|
const probe = new RedirectProbe(options);
|
|
474
621
|
try {
|
|
622
|
+
const observed = await outcome(probe.observe());
|
|
623
|
+
if (!observed.ok)
|
|
624
|
+
return {
|
|
625
|
+
incomplete: `redirect-secret: observer setup failed: ${observed.error.message}`,
|
|
626
|
+
};
|
|
475
627
|
await probe.setup();
|
|
476
628
|
if (options.mode === 'body') await probe.body();
|
|
477
629
|
else await probe.headers();
|