@ai-ecoverse/slicc-browsers 1.5.0 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +234 -5
- package/fixture-server.mjs +126 -0
- package/guard-driver.mjs +213 -0
- package/guard-runner.mjs +1035 -0
- package/guard-vectors.json +9198 -0
- package/package.json +21 -7
- package/schema/guard-vectors.schema.json +1687 -0
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@ The browser catalog, JSON schemas and conformance corpus shared by the two progr
|
|
|
4
4
|
- `slicc browsers` and `slicc launch` in [slicc-cli](https://github.com/ai-ecoverse/slicc-cli) (Go, `CGO_ENABLED=0`);
|
|
5
5
|
- `playwright-cli list` and `open` in [slicc-cdp](https://github.com/ai-ecoverse/slicc-cdp) (Rust, also built for `wasm32-wasip1`).
|
|
6
6
|
|
|
7
|
-
It also holds the [protected origins](#protected-origins): the origins browser control must never reach, which slicc-agent checks too.
|
|
7
|
+
It also holds the [protected origins](#protected-origins): the origins browser control must never reach, which slicc-agent checks too. It also holds the [guard vectors](#guard-vectors), which every CDP guard runs: bios, slicc-cdp, slicc-extension and slicc-cli.
|
|
8
8
|
|
|
9
9
|
They can't share code, so they share data. Three packages wrap the same files, at one version:
|
|
10
10
|
|
|
@@ -24,6 +24,8 @@ They can't share code, so they share data. Three packages wrap the same files, a
|
|
|
24
24
|
| `corpus.json` | every case in one file, generated from `corpus/` by `npm run corpus` |
|
|
25
25
|
| `protected.json`, `schema/protected.schema.json` | the protected-origin test vectors and their schema |
|
|
26
26
|
| `protected.mjs` | the reference protected-origin matcher, in JavaScript (npm only) |
|
|
27
|
+
| `guard-vectors.json`, `schema/guard-vectors.schema.json` | the guard vectors and their schema |
|
|
28
|
+
| `guard-driver.mjs` | the guard vectors' beacon and canary helpers, in JavaScript (npm only) |
|
|
27
29
|
|
|
28
30
|
## Go
|
|
29
31
|
|
|
@@ -38,6 +40,7 @@ b, ok := c.Browser("chrome")
|
|
|
38
40
|
- `(*Catalog).Browser(id)` and `(*Catalog).Priority(id)`, the index in `browsers` (its length for an unknown id).
|
|
39
41
|
- `CatalogJSON()`, `CatalogSchemaJSON()`, `BrowsersSchemaJSON()`, `CorpusJSON()`, `ProtectedJSON()` and `ProtectedSchemaJSON()` return copies of the embedded files.
|
|
40
42
|
- `(*Catalog).ProtectedOrigins` is `catalog.json`'s `protectedOrigins`. `Protected() (*ProtectedVectors, error)` parses `protected.json` (`Origins`, `URLs`, `Targets`, `CookieDomains`, `StorageKeys`; a target's `Opener` is `""`, `OpenerProtected`, `OpenerUnprotected` or `OpenerUnknown`).
|
|
43
|
+
- `Guard() (*GuardVectors, error)` parses `guard-vectors.json`; `GuardVectorsJSON()` and `GuardVectorsSchemaJSON()` return copies of the files. `Guard()` parses it into `SchemaVersion`, `SetVersion`, `Targets`, `Variants`, `Capabilities`, `Placeholders` and `Vectors`. A `GuardExpected` is `Checks` (names to booleans) or `Text`, and a `GuardObservation` is `Means`, `DependsOn` and `Requires`. The `GuardTarget*`, `GuardOrdinary`, `GuardSameSite`, `GuardKind*`, `GuardVia*`, `GuardScopeAny`, `GuardControl`, `GuardGuarded`, `GuardCap*` and `GuardSession*` constants name the values.
|
|
41
44
|
- `Corpus() ([]Case, error)` returns the cases. See [Running the corpus](#running-the-corpus) for the helpers.
|
|
42
45
|
|
|
43
46
|
## Rust
|
|
@@ -50,15 +53,17 @@ slicc-browsers = "1"
|
|
|
50
53
|
slicc-browsers = { version = "1", features = ["corpus"] }
|
|
51
54
|
```
|
|
52
55
|
|
|
53
|
-
- Always: `CATALOG`, `CATALOG_SCHEMA`, `BROWSERS_SCHEMA`, `CORPUS_SCHEMA` and `
|
|
56
|
+
- Always: `CATALOG`, `CATALOG_SCHEMA`, `BROWSERS_SCHEMA`, `CORPUS_SCHEMA`, `PROTECTED_SCHEMA` and `GUARD_VECTORS_SCHEMA`, each the file's text via `include_str!`, and `SCHEMA` (1). No dependencies.
|
|
54
57
|
- Feature `serde`: `Catalog` and the other types (`protected_origins` is a `ProtectedOrigins`), `Catalog::parse(&str)`, `catalog()` (parses `CATALOG`), `Catalog::browser` and `Catalog::priority`. It pulls in `serde` and `serde_json`.
|
|
55
|
-
- Feature `corpus` (implies `serde`): `CORPUS` and `
|
|
58
|
+
- Feature `corpus` (implies `serde`): `CORPUS`, `PROTECTED` and `GUARD_VECTORS`, the text of `corpus.json`, `protected.json` and `guard-vectors.json`; the module `corpus` with `corpus()`, `Case`, `Layout` and the helpers below; the module `protected` with `vectors()`, `Vectors` and the `OPENER_*` constants; and the module `guard` with `vectors()`, `Vectors`, `Vector`, `Step`, `Observation`, `Expected` (`Checks` or `Text`), and the `TARGET_*`, `ORDINARY*`, `KIND_*`, `VIA_*`, `SCOPE_ANY`, `CONTROL`, `GUARDED`, `CAP_*` and `SESSION_*` constants. Use it from `[dev-dependencies]` only; with resolver 2 it stays out of normal builds.
|
|
56
59
|
|
|
57
60
|
Every feature builds for `wasm32-wasip1`.
|
|
58
61
|
|
|
59
62
|
## npm
|
|
60
63
|
|
|
61
|
-
`import catalog from '@ai-ecoverse/slicc-browsers' with { type: 'json' }`. The exports are `.` (`catalog.json`), `./catalog.json`, `./corpus.json`, `./protected.json`, `./protected.mjs`, `./schema/catalog.schema.json`, `./schema/browsers.schema.json`, `./schema/corpus.schema.json` and `./schema/
|
|
64
|
+
`import catalog from '@ai-ecoverse/slicc-browsers' with { type: 'json' }`. The exports are `.` (`catalog.json`), `./catalog.json`, `./corpus.json`, `./protected.json`, `./protected.mjs`, `./guard-vectors.json`, `./guard-driver.mjs`, `./schema/catalog.schema.json`, `./schema/browsers.schema.json`, `./schema/corpus.schema.json`, `./schema/protected.schema.json` and `./schema/guard-vectors.schema.json`.
|
|
65
|
+
|
|
66
|
+
`guard-driver.mjs` is described under [Guard vectors](#guard-vectors).
|
|
62
67
|
|
|
63
68
|
`protected.mjs` is the reference matcher for [protected origins](#protected-origins): `import { protectedOrigins } from '@ai-ecoverse/slicc-browsers/protected.mjs'`, then `protectedOrigins(catalog.protectedOrigins, extraOrigins)` returns `{ url(text), target({ url, previous, opener }), cookieDomain(domain), storageKey(key) }`, each a boolean. It needs only the WHATWG `URL`, so it runs in Node and in a browser.
|
|
64
69
|
|
|
@@ -123,6 +128,230 @@ A same-site frame shares its parent's process and has no target of its own. Ever
|
|
|
123
128
|
|
|
124
129
|
**`protected.json`** (schema `schema/protected.schema.json`): `origins`, the extra origins every vector runs with; `urls` (`url`, `protected`); `targets` (`url`, `previous?`, whether the target was protected before; `opener?`, `protected`, `unprotected` or `unknown`, and no `opener` for a target without one; `protected`); `cookieDomains` (`domain`, `protected`); `storageKeys` (`key`, `protected`). `npm run validate` runs every vector through `protected.mjs`, and no vector uses a port from 9222 to 9229.
|
|
125
130
|
|
|
131
|
+
## Guard vectors
|
|
132
|
+
|
|
133
|
+
Every CDP guard runs the vectors in `guard-vectors.json` (schema `schema/guard-vectors.schema.json`): bios, slicc-cdp, slicc-extension and slicc-cli. The sources are listed in `sources`:
|
|
134
|
+
- slicc-cli's `protectVectors` at `2d5d981` (`test/integration/testdata/link/protect.js`), with the ids kept;
|
|
135
|
+
- slicc-cdp's `test/integration/guard-vectors.json` at `4907a45`, where its `E1`, `E2`, `E3`, `E5` and `B2` are renamed `E1.onload`, `E2.binding`, `E3.bypass-csp`, `E5.console` and `B2.headers`;
|
|
136
|
+
- the cert lane's E and B rows;
|
|
137
|
+
- `C5.continue-request` and `C5.continue-response`, added next to `C5.document`.
|
|
138
|
+
|
|
139
|
+
**Top level.**
|
|
140
|
+
- `schemaVersion` (1): guards pin it.
|
|
141
|
+
- `setVersion`: the package version. The release flow stamps it into the npm package and the crate (`scripts/set-crate-version.mjs`). In git, and so in the Go module, it reads `0.0.0`; Go code gets the version from the module.
|
|
142
|
+
- `targets`: the names a runner maps to a scheme, host and port: `seven`, `vault`, `vault-x` (a vault branch preview), `apex`, `own` (the client page's origin, `{client}`), `ordinary` and `ordinary-same-site`. Each has a `description` and, for the protected ones, a default `host`. Nothing in the file is a URL a runner must use.
|
|
143
|
+
- `variants`: the detach variants in ms. `prompt` and `lagging` are documented; a runner may add others, such as `lagging-8s`, and reports the name it ran.
|
|
144
|
+
- `capabilities`: what a transport may lack. `autoAttach` covers `Target.setAutoAttach`, `children` steps and child sessions; `browserSession` covers calls on the browser session; `browserContexts` covers `Target.createBrowserContext` and `Target.disposeBrowserContext`.
|
|
145
|
+
- `placeholders`: every `{name}` used in the vectors, with what it stands for. `{client}` is the client page's origin, which the runner maps.
|
|
146
|
+
|
|
147
|
+
**A vector** has:
|
|
148
|
+
- `id`, and `family` (`C`, `E` or `B`, the id's first letter);
|
|
149
|
+
- `kind`: `cdp` (runnable steps) or `text` (rows in words, which runners report as `not-run`, never `pass`);
|
|
150
|
+
- `scope`: `any`, or the one guard the vector is for;
|
|
151
|
+
- `setup` and `trigger`, both lists of steps;
|
|
152
|
+
- `targets`: the protected targets it runs against;
|
|
153
|
+
- `requires?`: the capabilities it needs, the union of its steps' and observations' `requires`. A runner whose transport lacks one reports the vector as `skip` with the reason `not-supported:<capability>`;
|
|
154
|
+
- `observe?`: each observed name, as `{ "means", "dependsOn"?, "requires"? }`. `dependsOn` lists the refusals the observation depends on (see Refusals);
|
|
155
|
+
- `live`: the liveness check;
|
|
156
|
+
- `control` (`origin: "ordinary"`, `via`, `expected`) and `expected`: an object of names and booleans, or text;
|
|
157
|
+
- `skip?` (`target`, `reason`): `reason` starts with a code, as in `not-applicable: …`.
|
|
158
|
+
|
|
159
|
+
**Steps** are declarative and transport-neutral. Any step can take `on` (`control` or `guarded`), which runs it on that leg only.
|
|
160
|
+
Any `cdp`, `page` or `children` step can take `requires`, the capabilities it needs: `autoAttach` for `Target.setAutoAttach`, `children` and `session: "child"`, `browserSession` for `session: "browser"`, and `browserContexts` for `Target.createBrowserContext` and `Target.disposeBrowserContext`.
|
|
161
|
+
- `{ "cdp": method, "params"?, "session"?, "as"?, "refusable"?, "refusal"? }`: a CDP call. It runs on the page session by default, on the browser with `browser`, on either with `page-or-browser` (the driver uses the page session), or on the current auto-attached target with `child`. `as` names its outcome. `refusable: true` with `refusal: { "match" }` (a regular expression for the error message or code) marks a call that a guard may refuse at registration.
|
|
162
|
+
- `{ "page": action, … }`:
|
|
163
|
+
- `tab`: a new tab at `url`, in `context`, which becomes the page. `background` defaults to `true`;
|
|
164
|
+
- `navigate`: go to `url` and wait for the load;
|
|
165
|
+
- `href`, `replace`, `form`, `post`: navigate from the page, 50 ms later;
|
|
166
|
+
- `open`: `window.open` with a user gesture;
|
|
167
|
+
- `iframe`: add an iframe;
|
|
168
|
+
- `framed`: load the `/framed` page from `from`, with `url` in an iframe;
|
|
169
|
+
- `fetch`, `xhr`, `image`: load `url` and name the body (`width N` for an image, or `<status> <body>` with `status`); `background` doesn't wait;
|
|
170
|
+
- `eval`: evaluate `expression`.
|
|
171
|
+
`hops` routes `url` through `/hop` on each `origin`, with its `code`.
|
|
172
|
+
- `{ "answer": { "url", "stage"?, "cdp"?, "params"?, "hold"? } }`: answer each paused request or auth challenge for `url` with that call; `hold` leaves it paused. Every other paused request continues unchanged.
|
|
173
|
+
- `{ "children": steps }`: run these steps on each target auto-attached from then on, then resume it.
|
|
174
|
+
- `{ "settle": { "beacon"?, "title"?, "ms"?, "variant"?, "session"? } }`: wait for a beacon from `beacon`, for the page `title`, or for the session to detach. It waits up to `ms` (10000 by default), then another 600 ms. `variant` waits the run's detach variant instead.
|
|
175
|
+
- `{ "wait": ms }`.
|
|
176
|
+
- `{ "guess": { "between", "cdp", "params" }, "as" }`: the call for each request id between the ids paused for the two URLs.
|
|
177
|
+
- `{ "text": … }`: a step in words.
|
|
178
|
+
|
|
179
|
+
Popups are observed by `Target.targetCreated` with the opener's `openerId`, or by the child's `Target.attachedToTarget`, never by the URL: a popup waiting for the debugger can still be `about:blank`.
|
|
180
|
+
|
|
181
|
+
**The stand-in** serves these routes on every origin. It never listens on 9222 to 9229.
|
|
182
|
+
- `/doc?n=`: an HTML page with the `data-canary` marker; `csp=1` adds `Content-Security-Policy: script-src 'none'` and an inline script that sends a beacon tagged `inline-<n>` to the log, which the CSP blocks unless it is bypassed;
|
|
183
|
+
- `/hop?code=&to=` and `/framed?src=`;
|
|
184
|
+
- `/data?n=`: `real-<n>`, with CORS;
|
|
185
|
+
- `/img`: a 1×1 PNG;
|
|
186
|
+
- `/auth`: Basic `u`:`p`, then `authed`;
|
|
187
|
+
- `/rewritten` (title `rewritten`);
|
|
188
|
+
- `/plant`: sets the canary;
|
|
189
|
+
- `/beacon`, and `/log?n=`, which returns `{ hits, beacons }`.
|
|
190
|
+
|
|
191
|
+
**The control rule.** Every vector has a control on an ordinary origin: the same setup and trigger against an unprotected origin must succeed. A missing or failing control fails the id. With `via: "guard"`, the control runs through the guard. With `via: "unguarded"`, the runner runs setup and trigger through its own CDP against the ordinary origin and against the protected stand-in, and both must show the effect. The runner reports the control's transport as `via`.
|
|
192
|
+
|
|
193
|
+
**Refusals.** Each refusable step has a refusal name: its `as`, or `refused.<i>`, where `i` is its index in the vector's steps, setup then trigger, with each `children` step's steps right after it, counted from 0 before `on` is applied. When a guard refuses the step and the error matches `refusal.match`, the runner sets that name to `true` in the leg's observations. On that leg, an observation whose `dependsOn` lists the name is skipped; every other observation is checked as usual. A guard that refuses the step on every origin passes in one of two ways: through the guard, with the step refused on both legs, or with an unguarded control and the guard's matched refusal on the guarded leg. With `via: "guard"`, a refusal on one leg only fails that leg. A guard that allows the step must pass its control through the guard.
|
|
194
|
+
|
|
195
|
+
**Liveness.** A runner checks `live` through its own CDP, never through the guard path: either the protected document loaded (title, marker or URL), or the guard returned its own refusal for a refusable step. A failing `live` fails the id.
|
|
196
|
+
|
|
197
|
+
**Design-dependent outcomes.** A key listed in `designDependent` may expect `"either"`, which `true` and `false` both match: a guard that refuses a setup step lets the real document load, and one that blocks before commit doesn't. The hard negatives stay required.
|
|
198
|
+
|
|
199
|
+
**Observability.** `observable.own: false` marks a vector the guard host's own origin can't show (headers, auth, redirects); a runner whose `own` is a static server reports it as `skip` with `not-observable`.
|
|
200
|
+
|
|
201
|
+
**Applies to.** `appliesTo` lists the triggers a vector runs with (`href`, `iframe`, `popup`, `redirect`). Chrome doesn't apply some session overrides inside a cross-site iframe, so those E6 rows list `href` only.
|
|
202
|
+
|
|
203
|
+
**Variants.** `variants` gives the delay in milliseconds before the client's `Target.detachFromTarget`: `prompt` 0, `lagging` 8000. A runner may add its own variant names.
|
|
204
|
+
|
|
205
|
+
**Reports.** There is one row per id, target, variant and transport. `control`, `guarded` and `live` are each `pass`, `fail`, `skip` (with the skip's `reason`) or `not-run` (for text rows):
|
|
206
|
+
|
|
207
|
+
```json
|
|
208
|
+
{ "id": "C1.script-nav", "target": "vault", "variant": "lagging", "transport": "cli", "via": "guard", "control": "pass", "guarded": "pass", "live": "pass" }
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
**`guard-driver.mjs`** has no dependencies and no imports. It uses no Node APIs, so it runs in a plain browser page as well as in Node. It drives a client of this shape:
|
|
212
|
+
- `send(method, params, sessionId)`: returns a promise of the result, and rejects with an error carrying `cdp`, the CDP error;
|
|
213
|
+
- `waitEvent(predicate, ms)`: resolves with the first matching event `{ method, params, sessionId }`;
|
|
214
|
+
- `events`: every event received so far.
|
|
215
|
+
|
|
216
|
+
It exports:
|
|
217
|
+
- `beacon(log, tag)` and `probe(log, tag)`: the sources for `{beacon}` and `{probe}`;
|
|
218
|
+
- `canary.plant(value)`, `canary.read()` and `canary.untouched(value, read)`: the canary in `localStorage`, IndexedDB and the stand-in's marker;
|
|
219
|
+
- `hop`, `expand(value, vars)`, and `steps(vector, side, vars)`, a leg's expanded steps, with `refusal.name` on each refusable step and `background` set on each `tab` step;
|
|
220
|
+
- `refusals(vector)`: the vector's refusal names;
|
|
221
|
+
- `outcome`, `call(client, step, sessionId)` and `isRefusal(outcome, step)`;
|
|
222
|
+
- `instrument(client, sessionId, log, tag)`, `load(client, sessionId, url)` and `popup(client, openerId)`;
|
|
223
|
+
- `judge(vector, side, observed)`: `{ pass, refused, skipped, mismatches }`, where `refused` lists the leg's refusals and `skipped` the observations that depend on them;
|
|
224
|
+
- `result(vector, run)`: a report row from `{ target, variant, transport, via?, capabilities?, control, guarded, live }`. `capabilities` is the transport's; without it, the transport has them all;
|
|
225
|
+
- `summary({ guard, sha, set, results })`: `set` is the parsed `guard-vectors.json`. It returns `{ set, schemaVersion, guard, sha, variants, targets, transports, results, skipped, notRun, missing, pass, line }`. `missing` lists each `{ id, target, variant, transport }` without a row, for every vector whose `scope` is `any` or the guard, every target it lists, every variant in `variants` or `results`, and every transport in `results`; `pass` needs it empty, where `line` is `GUARD-PASS <guard>@<sha> (vectors @ai-ecoverse/slicc-browsers@X.Y.Z)` or `GUARD-FAIL …`.
|
|
226
|
+
|
|
227
|
+
**`npm run validate`** checks that:
|
|
228
|
+
- the file matches the schema, and ids are unique;
|
|
229
|
+
- every vector has a control and an `expected`, text rows included;
|
|
230
|
+
- each id is in its family;
|
|
231
|
+
- a `cdp` vector has a `cdp` or `page` step, and a `text` vector has neither;
|
|
232
|
+
- every CDP method on the guarded leg also runs on the control;
|
|
233
|
+
- a skip names one of the vector's targets;
|
|
234
|
+
- `observe` defines every expected name;
|
|
235
|
+
- in a vector with a refusable step, every observation, except an `as` of a step before the first refusable one, has `dependsOn`, and each name in it is a refusable step's refusal name;
|
|
236
|
+
- every `Target.setAutoAttach`, `children`, child-session, browser-session and browser-context step has the matching `requires`, and a vector's `requires` is the union of its steps' and observations';
|
|
237
|
+
- every `{name}` is in `placeholders`;
|
|
238
|
+
- no URL uses a port from 9222 to 9229.
|
|
239
|
+
|
|
240
|
+
## Reference runner
|
|
241
|
+
|
|
242
|
+
`guard-runner.mjs` is dependency-free browser ESM. `run(set, options)` returns
|
|
243
|
+
`{ results, summary(metadata) }`; the module also exports `summary`. Results have
|
|
244
|
+
`id`, `target`, `variant`, `control`, `guarded`, `live`, boolean observations and
|
|
245
|
+
errors. Statuses are `pass`, `fail`, `skip` and `not-run`. Text vectors are reported
|
|
246
|
+
as `not-run`; they are not silently certified. A missing observation check is an
|
|
247
|
+
error before execution.
|
|
248
|
+
|
|
249
|
+
Supply a CDP client with `send(method, params, sessionId): Promise<object>`,
|
|
250
|
+
`waitEvent(predicate, milliseconds): Promise<event>` and an append-only `events`
|
|
251
|
+
array of `{ method, params, sessionId }`. Rejections should expose the protocol
|
|
252
|
+
error as `error.cdp`, or an error message. `client` carries the guarded leg;
|
|
253
|
+
`unguarded`, when supplied, carries unguarded controls and observation reads.
|
|
254
|
+
There is no fallback from observer reads to the guarded client. A bare client
|
|
255
|
+
means guarded transport only; rows requiring an observer report
|
|
256
|
+
`skip` with `not-supported:unguarded-observer`, counted in `summary().observerSkipped`.
|
|
257
|
+
`clientFactory({ variant, detachDelay, vector, target, slot })` is called per row
|
|
258
|
+
and may return either a client or `{ client, unguarded, close, transportVariant }`.
|
|
259
|
+
The factory can configure the guard's own transport delay from this metadata;
|
|
260
|
+
the runner never reaches into the guard. `transportVariant: true` acknowledges
|
|
261
|
+
that configuration, and every result records that boolean (false by default).
|
|
262
|
+
`slot` allows pooling browsers while keeping each row's clients independent.
|
|
263
|
+
The optional `close` releases the row's clients; release pooled browsers after
|
|
264
|
+
`run` completes. Each parallel slot needs an independent browser for
|
|
265
|
+
browser-wide settings. Without a factory execution is serial. Cookie and storage
|
|
266
|
+
rows use fresh browser contexts when `capabilities.browserContexts` is true;
|
|
267
|
+
otherwise they run serially. `capabilities` is an object of booleans or an array
|
|
268
|
+
of supported names. `capabilities.scope` selects owner-specific rows.
|
|
269
|
+
|
|
270
|
+
Map `ordinary`, `ordinary-same-site`, `seven`, `vault`, `vault-x`, `apex`, `own`
|
|
271
|
+
and `client` to origins. Optionally map `plain`, `cross` and `log`; their defaults
|
|
272
|
+
are `ordinary`, `client` and `ordinary`. Origins must preserve the site
|
|
273
|
+
relationships of the data. Tags and cookie names are unique per leg. `variant`
|
|
274
|
+
selects one variant or an array; omission selects all. `parallel` defaults to 1.
|
|
275
|
+
`log(row)` receives completed rows. A variant supplies the wait before an
|
|
276
|
+
explicit `Target.detachFromTarget` step and for a `settle.variant` step.
|
|
277
|
+
|
|
278
|
+
Every runnable vector has a declarative `liveCheck`; every observation has a
|
|
279
|
+
declarative `check`. Each check declares its `observer`: `fixture` reads request
|
|
280
|
+
logs, `client` reads already-delivered transport data, and `unguarded` requires
|
|
281
|
+
an independent CDP observer. `all` needs the observers for every operand; `any`
|
|
282
|
+
can use an available fixture or client alternative without an unguarded observer.
|
|
283
|
+
Unavailable optional observer alternatives are false, never substituted with guarded reads.
|
|
284
|
+
Checks include: `beacon`, `served`, `header` with
|
|
285
|
+
`on`, `event` with `where`, `result` with `path` and `equals`, `cookie` with `from`,
|
|
286
|
+
`compare`, `refused`, `reached`, `attached`, `delivered`, `title`, `expression`,
|
|
287
|
+
`hops` (each exact nested redirect URL), `detached`, `loaded`, `pre`, `blocked`,
|
|
288
|
+
`registration`, `state` and `filtered`. The diagnostic `executed` and `live`
|
|
289
|
+
checks are not valid liveness proofs.
|
|
290
|
+
Compose checks with `all`, `any` or `not`. Predicates support nested partial
|
|
291
|
+
objects, `$contains`, `$regex`, `$some`, nonempty `$every`, and `$encoded` with
|
|
292
|
+
`{ value, prefix }` for a base64-encoded value with an optional prefix. Reads of titles and
|
|
293
|
+
expressions use the unguarded observer. A `refused` check can declare `refusal.match` to match protocol errors. Transport timeouts, timeout sentinel values and evaluation exceptions fail result,
|
|
294
|
+
comparison and observer-read
|
|
295
|
+
observations; they are never comparable values. Registration refusals must match the
|
|
296
|
+
step's declared expression; dependent observations are omitted and reported by
|
|
297
|
+
the existing result evaluator.
|
|
298
|
+
|
|
299
|
+
`liveCheck` applies only to the guarded leg. Controls are judged by their positive
|
|
300
|
+
observations: a missing or failing effect fails the control. Guarded liveness
|
|
301
|
+
requires the exact target URL in the fixture log or a target-origin `/live`
|
|
302
|
+
heartbeat; alternatively a pre-action `/live?tag=...&step=pre&to=...` request and
|
|
303
|
+
an exact-URL block/error from the client or unguarded frame tree. An error for a
|
|
304
|
+
different URL is not evidence. A declared setup refusal can supply liveness only
|
|
305
|
+
when its `refusal.match` matched. Transport connectivity and command completion
|
|
306
|
+
alone never prove liveness. Cookie and storage rows seed a canary outside the
|
|
307
|
+
guard and read it back outside the guard; refused writes require unchanged
|
|
308
|
+
canary state and a refusal or filtered call response. Their control can clear or
|
|
309
|
+
modify state as required by its positive observations. `C5.auth` and
|
|
310
|
+
`C5.guessed-id.continue-with-auth` use fixture Authorization logs and refusal of
|
|
311
|
+
candidate auth ids; `C5.auth` also checks delivered auth events. Their liveness
|
|
312
|
+
requires an auth request at the fixture, not a completed fetch. Neither uses a
|
|
313
|
+
plain-tab baseline: headless Chromium can leave a credentials-including fetch
|
|
314
|
+
to a Basic 401 pending even without an attached client.
|
|
315
|
+
|
|
316
|
+
Beacon settlement uses only requests recorded after the page action starts. For
|
|
317
|
+
redirect chains it also requires the exact final destination request in the
|
|
318
|
+
fixture log. A setup beacon on the same origin and tag cannot settle an unfinished
|
|
319
|
+
redirect chain. This does not change the observations used to judge either leg.
|
|
320
|
+
|
|
321
|
+
The separately exported Node module `@ai-ecoverse/slicc-browsers/fixture` exposes
|
|
322
|
+
`fixtureServer({ host, port })`, defaulting to a random loopback port. It returns
|
|
323
|
+
`origin`, `port`, `log()`, `reset()` and `close()`. Log entries contain `url`,
|
|
324
|
+
`method`, `headers` and `time`. Mirror that API when using another fixture.
|
|
325
|
+
The server serves document, frame, redirect, CSP, authentication, data, image,
|
|
326
|
+
canary and beacon paths. Configure host resolution in your test browser, not in
|
|
327
|
+
the runner. `npm run test:guard` exercises Chromium twice with no guard, one target per
|
|
328
|
+
invocation: controls must pass and guarded expectations must fail. For a single
|
|
329
|
+
target use `GUARD_TARGET=seven npm run test:guard-target`. The fixture also serves
|
|
330
|
+
`/live` image requests from stand-in documents, independently of injected code.
|
|
331
|
+
|
|
332
|
+
```js
|
|
333
|
+
import set from '@ai-ecoverse/slicc-browsers/guard-vectors.json' with { type: 'json' };
|
|
334
|
+
import { run } from '@ai-ecoverse/slicc-browsers/guard-runner';
|
|
335
|
+
import { fixtureServer } from '@ai-ecoverse/slicc-browsers/fixture';
|
|
336
|
+
const fixture = await fixtureServer();
|
|
337
|
+
const targets = await configureTestOrigins(fixture.port);
|
|
338
|
+
const capabilities = {
|
|
339
|
+
autoAttach: true,
|
|
340
|
+
browserSession: true,
|
|
341
|
+
browserContexts: true,
|
|
342
|
+
};
|
|
343
|
+
const execution = await run(set, {
|
|
344
|
+
clientFactory: (metadata) => connectGuardAndObserver(metadata),
|
|
345
|
+
targets,
|
|
346
|
+
capabilities,
|
|
347
|
+
variant: ['prompt', 'lagging'],
|
|
348
|
+
parallel: 6,
|
|
349
|
+
fixture,
|
|
350
|
+
});
|
|
351
|
+
console.log(execution.summary({ guard: 'my-guard', sha: 'build-sha' }));
|
|
352
|
+
await fixture.close();
|
|
353
|
+
```
|
|
354
|
+
|
|
126
355
|
## The process source
|
|
127
356
|
|
|
128
357
|
The process scan finds browsers started with a debugging port by something else:
|
|
@@ -232,7 +461,7 @@ for case in corpus().cases {
|
|
|
232
461
|
|
|
233
462
|
Go 1.26, Rust stable with `wasm32-wasip1`, Node ≥ 24.
|
|
234
463
|
- `npm run corpus` regenerates `corpus.json` from `corpus/`; the pre-commit hook and CI fail when it's stale.
|
|
235
|
-
- `npm run validate` checks `catalog.json`, `corpus.json` and `
|
|
464
|
+
- `npm run validate` checks `catalog.json`, `corpus.json`, `protected.json` and `guard-vectors.json` against the schemas, runs the protected-origin vectors through `protected.mjs`, and checks what a schema can't say: unique ids, bundle ids and names, known Windows roots, rising tiers, compiling patterns, known tokens, tier order, `sevenOpen`, `six` and `auto` in each `expected.json`, and no port from 9222 to 9229.
|
|
236
465
|
- `npm run lint`: the slicc lint tools (Biome, no comments, no unit tests in git, `AGENTS.md`), `gofmt`, `go mod tidy -diff`, `go vet`, golangci-lint, `cargo fmt --check` and clippy.
|
|
237
466
|
- `npm test`: `validate`, the Go tests in `test/integration/`, and `cargo test --all-features`.
|
|
238
467
|
- `npm run build`: the Go module, the crate (also for `wasm32-wasip1`), and `npm pack --dry-run`.
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
import { createServer } from 'node:http';
|
|
2
|
+
|
|
3
|
+
const tall = Buffer.from(
|
|
4
|
+
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==',
|
|
5
|
+
'base64'
|
|
6
|
+
);
|
|
7
|
+
const escape = (value) =>
|
|
8
|
+
value.replaceAll('&', '&').replaceAll('"', '"').replaceAll('<', '<');
|
|
9
|
+
|
|
10
|
+
function document(url, html) {
|
|
11
|
+
const p = url.searchParams;
|
|
12
|
+
const csp = p.has('csp') || url.pathname === '/csp';
|
|
13
|
+
return html(
|
|
14
|
+
'doc',
|
|
15
|
+
`<p>document ${escape(p.get('n') ?? '')}</p>${csp ? '<script>document.title="inline"; document.body.dataset.inline="true"</script>' : ''}`,
|
|
16
|
+
csp ? { 'content-security-policy': "script-src 'none'" } : {}
|
|
17
|
+
);
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
function trigger(p) {
|
|
21
|
+
const kind = p.get('k') ?? '';
|
|
22
|
+
const to = p.get('to') ?? '';
|
|
23
|
+
const frame = kind === 'iframe' ? `<iframe src="${escape(to)}"></iframe>` : '';
|
|
24
|
+
return `<p>trigger</p><button onclick="go()">Go</button>${frame}<script>const to=${JSON.stringify(to)};function go(){const k=${JSON.stringify(kind)};if(k==='href'||k==='deferred')location.href=to;if(k==='replace')location.replace(to);if(k==='popup')window.open(to);if(k==='form'){const u=new URL(to);const f=document.createElement('form');f.action=u.origin+u.pathname;for(const [name,value] of u.searchParams){const i=document.createElement('input');i.type='hidden';i.name=name;i.value=value;f.append(i);}document.body.append(f);f.submit();}return 'went';}${kind === 'deferred' ? `setTimeout(go,${Number(p.get('defer') ?? 3000)});` : ''}</script>`;
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
function route(req, url, send, html) {
|
|
28
|
+
const p = url.searchParams;
|
|
29
|
+
const routes = {
|
|
30
|
+
'/beacon': () => send(204, ''),
|
|
31
|
+
'/hop': () =>
|
|
32
|
+
send(Number(p.get('code') ?? 302), 'moved', 'text/plain', { location: p.get('to') ?? '/' }),
|
|
33
|
+
'/data': () => send(200, `real-${p.get('n') ?? ''}`, 'text/plain'),
|
|
34
|
+
'/live': () => send(200, tall, 'image/png'),
|
|
35
|
+
'/img': () => send(200, tall, 'image/png'),
|
|
36
|
+
'/script': () =>
|
|
37
|
+
send(
|
|
38
|
+
200,
|
|
39
|
+
`globalThis.vectorScript=${JSON.stringify(`real-${p.get('n') ?? ''}`)}`,
|
|
40
|
+
'text/javascript'
|
|
41
|
+
),
|
|
42
|
+
'/page': () => html('trigger', trigger(p)),
|
|
43
|
+
'/api/cookies': () => send(200, req.headers.cookie ?? 'none', 'text/plain'),
|
|
44
|
+
'/plant': () =>
|
|
45
|
+
html('plant', '<script>localStorage.setItem("canary","planted")</script>', {
|
|
46
|
+
'set-cookie': [`canary=planted-${url.hostname}; Path=/`, 'del=1; Path=/'],
|
|
47
|
+
}),
|
|
48
|
+
'/rewritten': () => html('rewritten', '<p>REWRITTEN</p>'),
|
|
49
|
+
'/framed': () => {
|
|
50
|
+
const to = p.get('url') ?? 'about:blank';
|
|
51
|
+
const log = p.get('log') ?? url.origin;
|
|
52
|
+
const pre = `${log}/live?tag=${encodeURIComponent(p.get('tag') ?? '')}&step=pre&to=${encodeURIComponent(to)}`;
|
|
53
|
+
return html(
|
|
54
|
+
'framed',
|
|
55
|
+
`<script>navigator.sendBeacon(${JSON.stringify(pre)})</script><iframe src="${escape(to)}"></iframe>`
|
|
56
|
+
);
|
|
57
|
+
},
|
|
58
|
+
'/doc': () => document(url, html),
|
|
59
|
+
'/csp': () => document(url, html),
|
|
60
|
+
};
|
|
61
|
+
if (req.method === 'OPTIONS') return send(204, '');
|
|
62
|
+
if (url.pathname.startsWith('/auth')) {
|
|
63
|
+
if (req.headers.authorization === 'Basic dTpw') return send(200, 'authed', 'text/plain');
|
|
64
|
+
return send(401, 'unauthorized', 'text/plain', {
|
|
65
|
+
'www-authenticate': `Basic realm="${escape(url.pathname)}"`,
|
|
66
|
+
});
|
|
67
|
+
}
|
|
68
|
+
if (url.pathname.startsWith('/canary'))
|
|
69
|
+
return html(
|
|
70
|
+
'canary',
|
|
71
|
+
`<p data-canary="${escape(p.get('value') ?? 'planted')}">canary-marker ${escape(url.host)}</p>`
|
|
72
|
+
);
|
|
73
|
+
const handler = routes[url.pathname];
|
|
74
|
+
if (handler) return handler();
|
|
75
|
+
return send(404, 'missing', 'text/plain');
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
export async function fixtureServer({ host = '127.0.0.1', port = 0 } = {}) {
|
|
79
|
+
let entries = [];
|
|
80
|
+
const server = createServer((req, res) => {
|
|
81
|
+
const url = new URL(req.url, `http://${req.headers.host}`);
|
|
82
|
+
entries.push({
|
|
83
|
+
url: url.href,
|
|
84
|
+
method: req.method,
|
|
85
|
+
headers: { ...req.headers },
|
|
86
|
+
time: Date.now(),
|
|
87
|
+
});
|
|
88
|
+
const p = url.searchParams;
|
|
89
|
+
const cors = {
|
|
90
|
+
'access-control-allow-origin': req.headers.origin ?? '*',
|
|
91
|
+
'access-control-allow-credentials': 'true',
|
|
92
|
+
'access-control-allow-headers': '*',
|
|
93
|
+
'access-control-allow-methods': 'GET, POST, OPTIONS',
|
|
94
|
+
};
|
|
95
|
+
const send = (code, body, type = 'text/html', headers = {}) => {
|
|
96
|
+
res.writeHead(code, { 'content-type': `${type}; charset=utf-8`, ...cors, ...headers });
|
|
97
|
+
res.end(body);
|
|
98
|
+
};
|
|
99
|
+
const html = (title, body = '', headers = {}) =>
|
|
100
|
+
send(
|
|
101
|
+
200,
|
|
102
|
+
`<!doctype html><meta charset="utf-8"><title>${title}</title>${body}<img hidden src="/live?tag=${encodeURIComponent(p.get('tag') ?? p.get('n') ?? p.get('case') ?? p.get('value') ?? '')}&to=${encodeURIComponent(url.href)}">`,
|
|
103
|
+
'text/html',
|
|
104
|
+
headers
|
|
105
|
+
);
|
|
106
|
+
route(req, url, send, html);
|
|
107
|
+
});
|
|
108
|
+
await new Promise((resolve, reject) => {
|
|
109
|
+
server.once('error', reject);
|
|
110
|
+
server.listen(port, host, resolve);
|
|
111
|
+
});
|
|
112
|
+
const address = server.address();
|
|
113
|
+
return {
|
|
114
|
+
port: address.port,
|
|
115
|
+
origin: `http://${host}:${address.port}`,
|
|
116
|
+
log: () => structuredClone(entries),
|
|
117
|
+
reset: () => {
|
|
118
|
+
entries = [];
|
|
119
|
+
},
|
|
120
|
+
close: () =>
|
|
121
|
+
new Promise((resolve, reject) => {
|
|
122
|
+
server.closeAllConnections();
|
|
123
|
+
server.close((error) => (error ? reject(error) : resolve()));
|
|
124
|
+
}),
|
|
125
|
+
};
|
|
126
|
+
}
|
package/guard-driver.mjs
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
1
|
+
const q = JSON.stringify;
|
|
2
|
+
|
|
3
|
+
export const beacon = (log, tag) =>
|
|
4
|
+
`try { navigator.sendBeacon(${q(log)} + '/beacon?tag=' + ${q(tag)} + '&origin=' + encodeURIComponent(location.origin)); } catch {}`;
|
|
5
|
+
|
|
6
|
+
export const probe = (log, tag) =>
|
|
7
|
+
`(() => { ${beacon(log, tag)} const p = btoa(location.origin + location.pathname); if (window.probeBinding) window.probeBinding(p); console.log('probe:' + p); })()`;
|
|
8
|
+
|
|
9
|
+
const openCanary = `new Promise((resolve) => { const r = indexedDB.open('canary', 1); r.onupgradeneeded = () => r.result.createObjectStore('canary'); r.onsuccess = () => resolve(r.result); r.onerror = () => resolve(null); })`;
|
|
10
|
+
|
|
11
|
+
export const canary = {
|
|
12
|
+
plant: (value) =>
|
|
13
|
+
`(async () => { localStorage.setItem('canary', ${q(value)}); const db = await ${openCanary}; if (!db) return false; await new Promise((resolve) => { const t = db.transaction('canary', 'readwrite'); t.objectStore('canary').put(${q(value)}, 'canary'); t.oncomplete = resolve; t.onerror = resolve; }); db.close(); return true; })()`,
|
|
14
|
+
read: () =>
|
|
15
|
+
`(async () => { const db = await ${openCanary}; const record = db ? await new Promise((resolve) => { const g = db.transaction('canary').objectStore('canary').get('canary'); g.onsuccess = () => resolve(g.result ?? null); g.onerror = () => resolve(null); }) : null; db?.close(); return { localStorage: localStorage.getItem('canary'), indexedDB: record, marker: document.querySelector('[data-canary]')?.dataset.canary ?? null }; })()`,
|
|
16
|
+
untouched: (value, read) =>
|
|
17
|
+
read?.localStorage === value && read?.indexedDB === value && read?.marker === value,
|
|
18
|
+
};
|
|
19
|
+
|
|
20
|
+
export const hop = (hops, end) =>
|
|
21
|
+
hops.reduceRight(
|
|
22
|
+
(to, { origin, code }) => `${origin}/hop?code=${code}&to=${encodeURIComponent(to)}`,
|
|
23
|
+
end
|
|
24
|
+
);
|
|
25
|
+
|
|
26
|
+
export function expand(value, vars) {
|
|
27
|
+
if (typeof value === 'string')
|
|
28
|
+
return value.replace(/\{([A-Za-z][A-Za-z-]*)\}/g, (m, k) => (k in vars ? String(vars[k]) : m));
|
|
29
|
+
if (Array.isArray(value)) return value.map((v) => expand(v, vars));
|
|
30
|
+
if (value && typeof value === 'object')
|
|
31
|
+
return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, expand(v, vars)]));
|
|
32
|
+
return value;
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
function named(list, n = { i: 0 }) {
|
|
36
|
+
return list.map((s) => {
|
|
37
|
+
const i = n.i++;
|
|
38
|
+
const step = s.children ? { ...s, children: named(s.children, n) } : { ...s };
|
|
39
|
+
if (s.refusable) step.refusal = { ...s.refusal, name: s.as ?? `refused.${i}` };
|
|
40
|
+
if (s.page === 'tab') step.background = s.background ?? true;
|
|
41
|
+
return step;
|
|
42
|
+
});
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
const flatten = (list) => list.flatMap((s) => [s, ...flatten(s.children ?? [])]);
|
|
46
|
+
|
|
47
|
+
export const refusals = (vector) =>
|
|
48
|
+
flatten(named([...(vector.setup ?? []), ...(vector.trigger ?? [])]))
|
|
49
|
+
.filter((s) => s.refusable)
|
|
50
|
+
.map((s) => s.refusal.name);
|
|
51
|
+
|
|
52
|
+
export function steps(vector, side, vars) {
|
|
53
|
+
const pick = (list) =>
|
|
54
|
+
list
|
|
55
|
+
.filter((s) => !s.on || s.on === side)
|
|
56
|
+
.map(({ on: _, ...s }) => (s.children ? { ...s, children: pick(s.children) } : s));
|
|
57
|
+
const all = { ...vars, beacon: beacon(vars.log, vars.tag), probe: probe(vars.log, vars.tag) };
|
|
58
|
+
const [setup, trigger] = [vector.setup, vector.trigger].map((list, k, both) =>
|
|
59
|
+
named(list, { i: k === 0 ? 0 : flatten(both[0]).length })
|
|
60
|
+
);
|
|
61
|
+
return { setup: expand(pick(setup), all), trigger: expand(pick(trigger), all) };
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
export async function outcome(promise) {
|
|
65
|
+
try {
|
|
66
|
+
return { ok: true, result: await promise };
|
|
67
|
+
} catch (error) {
|
|
68
|
+
return { ok: false, error: error?.cdp ?? { message: String(error?.message ?? error) } };
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
export const isRefusal = (out, step) =>
|
|
73
|
+
Boolean(step?.refusal) &&
|
|
74
|
+
!out.ok &&
|
|
75
|
+
[out.error?.message, out.error?.code]
|
|
76
|
+
.filter((v) => v !== undefined && v !== null)
|
|
77
|
+
.some((v) => new RegExp(step.refusal.match).test(String(v)));
|
|
78
|
+
|
|
79
|
+
export function call(client, step, sessionId) {
|
|
80
|
+
const target = step.session === 'browser' ? undefined : sessionId;
|
|
81
|
+
return outcome(client.send(step.cdp, step.params ?? {}, target));
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
export async function instrument(client, sessionId, log, tag) {
|
|
85
|
+
await client.send('Page.enable', {}, sessionId);
|
|
86
|
+
await client.send(
|
|
87
|
+
'Page.addScriptToEvaluateOnNewDocument',
|
|
88
|
+
{ source: beacon(log, tag) },
|
|
89
|
+
sessionId
|
|
90
|
+
);
|
|
91
|
+
await client.send('Page.setBypassCSP', { enabled: true }, sessionId);
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
export async function load(client, sessionId, url, ms = 20000) {
|
|
95
|
+
const loaded = client.waitEvent(
|
|
96
|
+
(e) => e.method === 'Page.loadEventFired' && e.sessionId === sessionId,
|
|
97
|
+
ms
|
|
98
|
+
);
|
|
99
|
+
await client.send('Page.navigate', { url }, sessionId);
|
|
100
|
+
await loaded;
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
export function popup(client, openerId, ms = 10000) {
|
|
104
|
+
const isPopup = (e) =>
|
|
105
|
+
(e.method === 'Target.targetCreated' || e.method === 'Target.attachedToTarget') &&
|
|
106
|
+
e.params?.targetInfo?.openerId === openerId;
|
|
107
|
+
const seen = client.events.find(isPopup);
|
|
108
|
+
if (seen) return Promise.resolve(seen.params.targetInfo);
|
|
109
|
+
return client.waitEvent(isPopup, ms).then(
|
|
110
|
+
(e) => e.params.targetInfo,
|
|
111
|
+
() => null
|
|
112
|
+
);
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
const matches = (want, got) => (want === 'either' ? typeof got === 'boolean' : got === want);
|
|
116
|
+
|
|
117
|
+
export function judge(vector, side, observed) {
|
|
118
|
+
const want = side === 'control' ? vector.control.expected : vector.expected;
|
|
119
|
+
const refused = refusals(vector).filter((name) => observed?.[name] === true);
|
|
120
|
+
if (typeof want === 'string') {
|
|
121
|
+
const mismatches = observed === true ? [] : [want];
|
|
122
|
+
return { pass: mismatches.length === 0, refused, skipped: [], mismatches };
|
|
123
|
+
}
|
|
124
|
+
const skipped = Object.keys(want).filter((k) =>
|
|
125
|
+
(vector.observe?.[k]?.dependsOn ?? []).some((d) => refused.includes(d))
|
|
126
|
+
);
|
|
127
|
+
const mismatches = Object.entries(want)
|
|
128
|
+
.filter(([k, v]) => !skipped.includes(k) && !matches(v, observed?.[k]))
|
|
129
|
+
.map(([k, v]) => `${k}: got ${q(observed?.[k])}, want ${q(v)}`);
|
|
130
|
+
return { pass: mismatches.length === 0, refused, skipped, mismatches };
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
export function result(vector, run) {
|
|
134
|
+
const row = {
|
|
135
|
+
id: vector.id,
|
|
136
|
+
target: run.target,
|
|
137
|
+
variant: run.variant,
|
|
138
|
+
transport: run.transport,
|
|
139
|
+
via: run.via ?? vector.control.via,
|
|
140
|
+
};
|
|
141
|
+
if (vector.kind === 'text')
|
|
142
|
+
return { ...row, control: 'not-run', guarded: 'not-run', live: 'not-run' };
|
|
143
|
+
if (vector.skip?.target === run.target)
|
|
144
|
+
return { ...row, control: 'skip', guarded: 'skip', live: 'skip', reason: vector.skip.reason };
|
|
145
|
+
const missing = (vector.requires ?? []).find(
|
|
146
|
+
(cap) => Array.isArray(run.capabilities) && !run.capabilities.includes(cap)
|
|
147
|
+
);
|
|
148
|
+
if (missing) {
|
|
149
|
+
const reason = `not-supported:${missing}`;
|
|
150
|
+
return { ...row, control: 'skip', guarded: 'skip', live: 'skip', reason };
|
|
151
|
+
}
|
|
152
|
+
const c = run.control ? judge(vector, 'control', run.control) : null;
|
|
153
|
+
const g = run.guarded ? judge(vector, 'guarded', run.guarded) : null;
|
|
154
|
+
const only = (a, b) => (a?.refused ?? []).filter((name) => !(b?.refused ?? []).includes(name));
|
|
155
|
+
const why = [];
|
|
156
|
+
let control = c?.pass ? 'pass' : 'fail';
|
|
157
|
+
if (!c) why.push('no control');
|
|
158
|
+
else if (c.refused.length > 0 && row.via !== 'guard') {
|
|
159
|
+
control = 'fail';
|
|
160
|
+
why.push('an unguarded control was refused');
|
|
161
|
+
} else if (row.via === 'guard' && g && only(c, g).length > 0) {
|
|
162
|
+
control = 'fail';
|
|
163
|
+
why.push(`refused only on the control: ${only(c, g).join(', ')}`);
|
|
164
|
+
} else why.push(...c.mismatches.map((m) => `control ${m}`));
|
|
165
|
+
let guarded = g?.pass ? 'pass' : 'fail';
|
|
166
|
+
if (!g) why.push('no guarded run');
|
|
167
|
+
else if (row.via === 'guard' && only(g, c).length > 0) {
|
|
168
|
+
guarded = 'fail';
|
|
169
|
+
why.push(`refused only on the protected target: ${only(g, c).join(', ')}`);
|
|
170
|
+
} else if (row.via === 'unguarded' && g.refused.length > 0 && !c?.pass) {
|
|
171
|
+
guarded = 'fail';
|
|
172
|
+
why.push('refused on the protected target without a passing control');
|
|
173
|
+
} else why.push(...g.mismatches);
|
|
174
|
+
const live = run.live === true ? 'pass' : 'fail';
|
|
175
|
+
if (live === 'fail') why.push('not live');
|
|
176
|
+
return { ...row, control, guarded, live, ...(why.length ? { why: why.join('; ') } : {}) };
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
const passed = (r) =>
|
|
180
|
+
[r.control, r.guarded, r.live].every((s) => s === 'pass') ||
|
|
181
|
+
[r.control, r.guarded, r.live].every((s) => s === 'skip') ||
|
|
182
|
+
[r.control, r.guarded, r.live].every((s) => s === 'not-run');
|
|
183
|
+
|
|
184
|
+
export function summary({ guard, sha, set, results }) {
|
|
185
|
+
const uniq = (k) => [...new Set(results.map((r) => r[k]))].sort();
|
|
186
|
+
const variants = [...new Set([...Object.keys(set.variants ?? {}), ...uniq('variant')])].sort();
|
|
187
|
+
const has = new Set(results.map((r) => q([r.id, r.target, r.variant, r.transport])));
|
|
188
|
+
const missing = set.vectors
|
|
189
|
+
.filter((v) => v.scope === 'any' || v.scope === guard)
|
|
190
|
+
.flatMap((v) => v.targets.map((target) => ({ id: v.id, target })))
|
|
191
|
+
.flatMap((row) => variants.map((variant) => ({ ...row, variant })))
|
|
192
|
+
.flatMap((row) => uniq('transport').map((transport) => ({ ...row, transport })))
|
|
193
|
+
.filter((r) => !has.has(q([r.id, r.target, r.variant, r.transport])));
|
|
194
|
+
const pass = results.length > 0 && missing.length === 0 && results.every(passed);
|
|
195
|
+
const name = `@ai-ecoverse/slicc-browsers@${set.setVersion}`;
|
|
196
|
+
return {
|
|
197
|
+
set: name,
|
|
198
|
+
schemaVersion: set.schemaVersion,
|
|
199
|
+
guard,
|
|
200
|
+
sha,
|
|
201
|
+
variants: uniq('variant'),
|
|
202
|
+
targets: uniq('target'),
|
|
203
|
+
transports: uniq('transport'),
|
|
204
|
+
results,
|
|
205
|
+
skipped: results
|
|
206
|
+
.filter((r) => r.control === 'skip')
|
|
207
|
+
.map(({ id, target, reason }) => ({ id, target, reason })),
|
|
208
|
+
notRun: [...new Set(results.filter((r) => r.control === 'not-run').map((r) => r.id))],
|
|
209
|
+
missing,
|
|
210
|
+
pass,
|
|
211
|
+
line: `${pass ? 'GUARD-PASS' : 'GUARD-FAIL'} ${guard}@${sha} (vectors ${name})`,
|
|
212
|
+
};
|
|
213
|
+
}
|