recovery-probe 0.4.0-preview.2 → 0.4.0-preview.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/CONTRIBUTING.md +6 -1
- package/README.md +39 -187
- package/bin/ipc.mjs +10 -9
- package/bin/refresh.mjs +9 -9
- package/docs/api.md +57 -0
- package/docs/observe.md +1 -1
- package/docs/runner.md +40 -0
- package/package.json +10 -5
- package/src/desktop-report.mjs +30 -28
- package/src/discovery-report.mjs +10 -11
- package/src/fetch-fault.mjs +10 -2
- package/src/ipc-probe.cjs +10 -5
- package/src/observation-report.mjs +52 -18
- package/src/report-template.mjs +33 -0
- package/src/report-templates/desktop.html +134 -0
- package/src/report-templates/discovery.html +46 -0
- package/src/report-templates/ipc.html +30 -0
- package/src/report-templates/observation.html +74 -0
- package/src/report-templates/refresh.html +28 -0
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,17 @@ All notable changes to Recovery Probe are documented here. The project follows [
|
|
|
4
4
|
|
|
5
5
|
Version 0.2.0 is the first npm release. Earlier 0.1.x revisions were repository-only prototypes.
|
|
6
6
|
|
|
7
|
+
## 0.4.0-preview.4 (2026-10-15)
|
|
8
|
+
|
|
9
|
+
- `createFaultFetch` passes relative URLs, such as `fetch('/api/x')`, to the supplied fetch untouched. Previously they threw `Invalid URL`.
|
|
10
|
+
- The HTML reports are built from template files in `src/report-templates`. Text slots are escaped, a missing slot throws, and the rendered text is unchanged.
|
|
11
|
+
- The README leads with one idea, whether apps show the right data, and with the demo. It drops from 2,380 to about 1,070 words. The fault kinds and library API move to `docs/api.md`, and the JSON-configured runner to `docs/runner.md`.
|
|
12
|
+
|
|
13
|
+
## 0.4.0-preview.3 (2026-10-15)
|
|
14
|
+
|
|
15
|
+
- Restore an existing instance-level `webContents.send` (for example an app's tracing wrapper) when an observation ends or the hook is disposed, instead of deleting it. Previously that wrapper was lost and later pushes bypassed it. A wrapper installed after the observation began is left in place.
|
|
16
|
+
- Broaden the project description: recovery after failed reads, cached content, and screens that lag behind their data. Observation and `DELAYED_UI_UPDATE` remain fault-free leads, not verdicts.
|
|
17
|
+
|
|
7
18
|
## 0.4.0-preview.2 (2026-10-15)
|
|
8
19
|
|
|
9
20
|
- Apply consistent Prettier formatting and add formatting, ESLint and JavaScript type checks to CI.
|
package/CONTRIBUTING.md
CHANGED
|
@@ -20,9 +20,14 @@ npm test
|
|
|
20
20
|
npm run test:package
|
|
21
21
|
npx playwright install chromium
|
|
22
22
|
npm run test:browser
|
|
23
|
+
npm run test:repair # safety, authenticated portal, before/after pipeline
|
|
24
|
+
# After installing the optional Electron test runtime:
|
|
25
|
+
npm run test:repair:electron
|
|
23
26
|
```
|
|
24
27
|
|
|
25
|
-
|
|
28
|
+
Repair integration tests use a **controlled mock model response** but real apps, fault injection, source changes and reruns. They validate orchestration, not live AI diagnosis quality.
|
|
29
|
+
|
|
30
|
+
Code and JSON are formatted with Prettier (`npm run format`) and linted with ESLint (`npm run lint`); CI fails on either. Functions passed to `page.evaluate` run in the browser: keep them self-contained, and list new in-page modules in `eslint.config.mjs` so they get browser globals. TypeScript checks the JavaScript in `src`, `bin` and `examples` (`npm run typecheck`, `checkJs` without emitting); add JSDoc types where inference is not enough. Strict mode is off for now and can be tightened file by file. Markdown, HTML fixtures, report templates and workflow files are left as written. Report markup lives in `src/report-templates`: `{{slot}}` takes text and escapes it, `{{{slot}}}` takes markup built with `escapeHtml`. `.gitattributes` keeps LF line endings on every platform so local checks match CI. The repository-wide reformat is listed in `.git-blame-ignore-revs`; run `git config blame.ignoreRevsFile .git-blame-ignore-revs` to skip it in local blame.
|
|
26
31
|
|
|
27
32
|
## Install an unpublished source preview
|
|
28
33
|
|
package/README.md
CHANGED
|
@@ -1,86 +1,55 @@
|
|
|
1
1
|
# Recovery Probe
|
|
2
2
|
|
|
3
|
-
**Test
|
|
3
|
+
**Test whether web and desktop apps show the right data.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Apps can show the wrong data without any error: a list that disappears after one failed read, a cached screen that never picks up a change, or a row whose state lags behind data the app already received. Recovery Probe reproduces these cases on demand. It injects bounded faults into data reads and compares the screen with a healthy run, or it watches reads and the screen without any fault. Each run writes a local report that separates what it measured from what it suspects.
|
|
6
|
+
|
|
7
|
+
Installing the package does not add recovery behavior to your application.
|
|
6
8
|
|
|
7
9
|
[](https://github.com/BojanKovachki/recovery-probe/actions/workflows/validate.yml)
|
|
8
10
|
[](https://www.npmjs.com/package/recovery-probe)
|
|
9
11
|
[](LICENSE)
|
|
10
12
|
|
|
11
|
-
##
|
|
12
|
-
|
|
13
|
-
| Application path | What can be tested | Setup |
|
|
14
|
-
| --- | --- | --- |
|
|
15
|
-
| Web apps in Chromium-based browsers | Renderer GET fetch/XHR failures, UI recovery and structural content changes | Launch a test browser or attach to an authorized development browser |
|
|
16
|
-
| Electron desktop apps using renderer HTTP | The same request and UI checks | Attach through a loopback debugging endpoint |
|
|
17
|
-
| Electron desktop apps using IPC reads | Renderer behavior after selected IPC rejections or null results | Add a development-only handler-registration hook and enable local inspection |
|
|
18
|
-
| JavaScript code using fetch | Recovery logic around an explicitly supplied fetch wrapper | Import `recovery-probe/fetch`; Playwright is not required |
|
|
13
|
+
## See it work
|
|
19
14
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
## Install
|
|
15
|
+
The package includes a deliberately broken app and a corrected one. Both pass the normal loading check; only the corrected one recovers from all three faults.
|
|
23
16
|
|
|
24
|
-
|
|
17
|
+

|
|
25
18
|
|
|
26
19
|
```bash
|
|
27
20
|
mkdir recovery-probe-tools && cd recovery-probe-tools
|
|
28
|
-
npm view recovery-probe@0.3.0 version
|
|
29
21
|
npm install --save-dev --save-exact recovery-probe@0.3.0 playwright@1.62.1
|
|
30
|
-
npx
|
|
22
|
+
npx playwright install chromium
|
|
23
|
+
npx recovery-probe --demo
|
|
31
24
|
```
|
|
32
25
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
For the 0.4 observation preview, install `recovery-probe@0.4.0-preview.2` (the `next` channel) with the same Playwright version. Preview.2 adds formatting, lint and implementation type checks; the action/row observation features were introduced in preview.1.
|
|
26
|
+
`--demo` exits successfully only when it observes that contrast: the happy path passes, the broken recovery is caught, and the corrected flow passes. Add `--json` to print the full report or `--out report.json` to save it.
|
|
36
27
|
|
|
37
|
-
##
|
|
28
|
+
## Test your own app
|
|
38
29
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
Content comparisons preserve the first observed result and summarize later samples: a temporary difference can subsequently match the baseline, or diverge again. These are observations, not ownership, freshness or stability verdicts. See the guide for updating the development IPC hook before capturing cached state; see the changelog for upgrading from 0.2.0.
|
|
42
|
-
|
|
43
|
-
| Goal | Entry point | Guide |
|
|
30
|
+
| Goal | Start with | Guide |
|
|
44
31
|
| --- | --- | --- |
|
|
45
|
-
| Set up a first
|
|
46
|
-
| Discover and test
|
|
32
|
+
| Set up a first recovery check interactively | `recovery-probe start web URL` or `start desktop` | [Guided start](docs/start.md) |
|
|
33
|
+
| Discover and test the reads behind a screen in a browser or Electron window | `recovery-probe desktop --cdp URL` | [Attachment and HTTP discovery](docs/desktop.md) |
|
|
47
34
|
| Compare an Electron screen under IPC faults | `recovery-probe ipc --discover --config FILE` | [IPC setup and discovery](docs/ipc.md) |
|
|
48
35
|
| Test a refresh after content has already loaded | `recovery-probe refresh --config FILE --json` | [Refresh experiments](docs/refresh.md) |
|
|
49
|
-
|
|
|
50
|
-
|
|
|
51
|
-
|
|
|
52
|
-
| Add an explicit recovery assertion to a test | `withFault` or `createFaultFetch` | Examples below |
|
|
36
|
+
| Watch cached content and row states after user actions, without faults | `recovery-probe observe --config FILE [--steps FILE]` | [Observation](docs/observe.md) |
|
|
37
|
+
| Run a repeatable recovery check in CI | `npm run test:ci-example` (source checkout) | [CI example](docs/ci.md) |
|
|
38
|
+
| Assert recovery inside your own tests | `withFault` or `createFaultFetch` | [API](docs/api.md) |
|
|
53
39
|
| Propose and verify a source change | Opt-in repair workflow | [Repair setup and limits](docs/repair.md) |
|
|
54
40
|
|
|
55
|
-
`desktop`
|
|
56
|
-
|
|
57
|
-
Guided setup asks you to choose a request and loaded content that proves success; later runs reuse the saved scenario. Structural discovery compares healthy and faulted screens without requiring a hand-written content selector, but still needs a selected read target and a valid baseline. Neither workflow infers your product's intended recovery policy.
|
|
58
|
-
|
|
59
|
-
HTTP and IPC structural discovery calibrate three healthy reloads for the full `baselineTimeoutMs` window before injecting faults (45 seconds with the default 15-second window). All three stable end states must agree. This avoids treating a briefly stable loading shell as the healthy screen, accepts legitimate empty regions, and keeps an unsuccessful calibration inconclusive. Later controls may finish early once they match the calibrated baseline. Content arriving after the window remains outside the evidence.
|
|
60
|
-
|
|
61
|
-
**Session-isolation boundary:** recovery/discovery/refresh workflows reload the renderer to establish controls. That clears in-memory caches and can hide state carried between sessions. Those modes cannot prove account isolation; content preservation within one session is not an ownership check. The new observation workflow collects evidence without resetting that state, but does not infer identities or data ownership.
|
|
62
|
-
|
|
63
|
-
## Refresh experiments
|
|
64
|
-
|
|
65
|
-
Refresh mode tests an **already populated screen** using an explicitly configured synthetic window event, such as `online`. It first checks the event without a fault, then injects the next selected read failure and observes the screen without reloading. An optional second event tests whether another refresh repairs the screen.
|
|
66
|
-
|
|
67
|
-
```bash
|
|
68
|
-
npx recovery-probe refresh --config ./refresh.json --out ./refresh-run1 --json
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
This dispatches an event; it does not disconnect the network, change `navigator.onLine`, simulate sleep/resume or expire authentication. Spontaneous recovery and recovery after the second tool event are reported separately. See the [configuration, results and cleanup contract](docs/refresh.md).
|
|
41
|
+
`desktop` also attaches to an existing Chromium-based web browser.
|
|
72
42
|
|
|
73
|
-
|
|
43
|
+
**Where it works:**
|
|
74
44
|
|
|
75
|
-
|
|
45
|
+
- **Web apps** in Chromium-based browsers: launch a test browser or attach to an authorized development one.
|
|
46
|
+
- **Electron apps that read over HTTP** in the renderer: attach through a loopback debugging endpoint.
|
|
47
|
+
- **Electron apps that read over IPC**: add a development-only hook to the main process.
|
|
48
|
+
- **JavaScript code that takes a fetch function**: use `recovery-probe/fetch`, without Playwright.
|
|
76
49
|
|
|
77
|
-
|
|
50
|
+
Desktop support means Electron, not arbitrary native applications. IPC faults do not exercise native networking, HTTP status handling or token refresh. An unsupported data path is inconclusive, never a successful recovery check.
|
|
78
51
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-

|
|
82
|
-
|
|
83
|
-
## Use in a Playwright test
|
|
52
|
+
## Add a recovery check to a Playwright test
|
|
84
53
|
|
|
85
54
|
Use `withFault` inside an existing Playwright test. The callback contains the real user action and the application-specific recovery assertion.
|
|
86
55
|
|
|
@@ -104,154 +73,37 @@ test('profile recovers after a transient 503', async ({ page }) => {
|
|
|
104
73
|
});
|
|
105
74
|
```
|
|
106
75
|
|
|
107
|
-
If
|
|
108
|
-
|
|
109
|
-
## Run the demonstration
|
|
110
|
-
|
|
111
|
-
The repository includes deliberately broken and corrected applications. Both pass their normal loading check; only the corrected application recovers from all three faults.
|
|
112
|
-
|
|
113
|
-
```bash
|
|
114
|
-
npm install
|
|
115
|
-
npx playwright install chromium
|
|
116
|
-
npx recovery-probe --demo
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
The command prints a concise comparison:
|
|
120
|
-
|
|
121
|
-
```text
|
|
122
|
-
broken synthetic fixture
|
|
123
|
-
PASS Normal connection (OK)
|
|
124
|
-
CAUGHT HTTP 503 once (RECOVERY_ASSERTION_FAILED, injected 1)
|
|
125
|
-
CAUGHT Interrupted request once (RECOVERY_ASSERTION_FAILED, injected 1)
|
|
126
|
-
CAUGHT Malformed JSON once (RECOVERY_ASSERTION_FAILED, injected 1)
|
|
127
|
-
|
|
128
|
-
fixed synthetic fixture
|
|
129
|
-
PASS Normal connection (OK)
|
|
130
|
-
PASS HTTP 503 once (OK, injected 1)
|
|
131
|
-
PASS Interrupted request once (OK, injected 1)
|
|
132
|
-
PASS Malformed JSON once (OK, injected 1)
|
|
133
|
-
```
|
|
76
|
+
If no request matches, `withFault` throws `FAULT_NOT_TRIGGERED` instead of letting the test pass silently. Code that takes a fetch function can use `createFaultFetch` instead. See the [API reference](docs/api.md) and the [JSON-configured runner](docs/runner.md).
|
|
134
77
|
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
## Library HTTP faults
|
|
138
|
-
|
|
139
|
-
| Kind | Injected behavior |
|
|
140
|
-
| --- | --- |
|
|
141
|
-
| `http-error` | Fulfils the matched request with an HTTP error status; defaults to `503`. |
|
|
142
|
-
| `connection-failure` | Aborts the matched Playwright request with `connectionfailed`, or throws from the fetch adapter. |
|
|
143
|
-
| `invalid-json` | Returns status `200` with a deliberately malformed JSON body. |
|
|
144
|
-
|
|
145
|
-
Faults are bounded by `count` (default `1`, maximum `20`). The count is reserved synchronously, so concurrent matching requests cannot over-inject.
|
|
146
|
-
|
|
147
|
-
## Fetch-only tests
|
|
148
|
-
|
|
149
|
-
The lightweight `recovery-probe/fetch` entry point has no runtime dependencies and does not require Playwright. It matches one exact absolute URL, including its query string, and never patches global `fetch`.
|
|
150
|
-
|
|
151
|
-
```js
|
|
152
|
-
import { createFaultFetch } from 'recovery-probe/fetch';
|
|
153
|
-
|
|
154
|
-
const probe = createFaultFetch(fetch, {
|
|
155
|
-
url: 'http://127.0.0.1:3000/api/profile',
|
|
156
|
-
kind: 'connection-failure',
|
|
157
|
-
});
|
|
158
|
-
|
|
159
|
-
await exerciseApplication(probe.fetch);
|
|
160
|
-
probe.assertApplied();
|
|
161
|
-
```
|
|
162
|
-
|
|
163
|
-
`exerciseApplication` represents your own test adapter or application controller. Calling `assertApplied()` is essential: it proves that every requested fault was consumed.
|
|
164
|
-
|
|
165
|
-
## Original JSON-configured runner
|
|
166
|
-
|
|
167
|
-
The CLI supports pages that automatically load GET data, expose a visible Retry button after failure, and show a ready element after recovery.
|
|
168
|
-
|
|
169
|
-
```json
|
|
170
|
-
{
|
|
171
|
-
"name": "Profile recovery",
|
|
172
|
-
"url": "http://127.0.0.1:3000/profile",
|
|
173
|
-
"requestPattern": "**/api/profile",
|
|
174
|
-
"readySelector": "[data-testid=profile-ready]",
|
|
175
|
-
"retrySelector": "[data-testid=profile-retry]",
|
|
176
|
-
"timeoutMs": 3000
|
|
177
|
-
}
|
|
178
|
-
```
|
|
179
|
-
|
|
180
|
-
```bash
|
|
181
|
-
npx recovery-probe --config recovery-probe.json --out artifacts/profile-recovery.json
|
|
182
|
-
```
|
|
183
|
-
|
|
184
|
-
The runner first checks the normal baseline in a fresh browser context. If that baseline fails, fault comparisons are skipped and reported as inconclusive. Each fault then receives its own fresh context with service workers blocked so Playwright can intercept the request.
|
|
185
|
-
|
|
186
|
-
Exit codes:
|
|
187
|
-
|
|
188
|
-
- `0`: every configured check passed, or the controlled demo observed its expected broken-versus-fixed result.
|
|
189
|
-
- `1`: a recovery check failed, was inconclusive, or was skipped.
|
|
190
|
-
- `2`: invalid arguments, configuration, or environment setup.
|
|
191
|
-
|
|
192
|
-
## Original runner result semantics
|
|
193
|
-
|
|
194
|
-
| Code | Meaning |
|
|
195
|
-
| --- | --- |
|
|
196
|
-
| `OK` | The configured assertion completed and the expected fault count was observed. |
|
|
197
|
-
| `RECOVERY_ASSERTION_FAILED` | The fault occurred, but the recovery assertion did not pass. |
|
|
198
|
-
| `FAULT_NOT_TRIGGERED` | The requested fault count was not observed. Check the route, method, cache, and service worker. |
|
|
199
|
-
| `BASELINE_FAILED` | The ordinary flow failed, so the runner could not make a useful fault comparison. |
|
|
200
|
-
| `INJECTION_ERROR` | Playwright could not apply or clean up the route reliably. |
|
|
201
|
-
|
|
202
|
-
A passing result proves the configured recovery behavior for that test. It is not a certification of overall application resilience.
|
|
203
|
-
|
|
204
|
-
## API
|
|
205
|
-
|
|
206
|
-
### `withFault(page, options, exercise)`
|
|
78
|
+
## Install
|
|
207
79
|
|
|
208
|
-
|
|
80
|
+
**0.3.0** is the stable release on npm's `latest` channel, and the commands above pin it for reproducible reports. The 0.4 observation preview is `recovery-probe@0.4.0-preview.4` on the `next` channel; install it with the same Playwright version.
|
|
209
81
|
|
|
210
|
-
|
|
211
|
-
interface FaultOptions {
|
|
212
|
-
match: string | RegExp;
|
|
213
|
-
kind: 'http-error' | 'connection-failure' | 'invalid-json';
|
|
214
|
-
status?: number;
|
|
215
|
-
count?: number;
|
|
216
|
-
method?: string;
|
|
217
|
-
}
|
|
218
|
-
```
|
|
82
|
+
Use Node.js 22 or newer. A separate tools folder keeps the tester out of your application's production dependencies. Browser and desktop commands need Playwright; attached targets use their existing browser runtime. If a version is not on npm yet, use the [source installation](CONTRIBUTING.md#install-an-unpublished-source-preview): a GitHub merge alone does not publish the package. See [CHANGELOG.md](CHANGELOG.md) for version history.
|
|
219
83
|
|
|
220
|
-
|
|
84
|
+
## Reports, agents and privacy
|
|
221
85
|
|
|
222
|
-
|
|
86
|
+
Reports separate measured facts, such as fault counts, successful reads, timings and UI changes, from suspected defects. A `CONTENT_LOSS` finding needs review. An unconsumed fault, an unstable control or an unsupported path is inconclusive. Observation findings such as `DELAYED_UI_UPDATE` are leads to check against the application logic, not verdicts.
|
|
223
87
|
|
|
224
|
-
|
|
88
|
+
Coding agents can run saved scenarios, read the JSON evidence, propose a change and rerun the same experiment. Recovery Probe supplies repeatable experiments and evidence; it is not a general bug detector or a production self-healing system.
|
|
225
89
|
|
|
226
|
-
|
|
90
|
+
Reports stay local, and Recovery Probe sends no telemetry. Scenarios, endpoints, channels, selected text, screenshots, login state and command logs may be private, so do not publish them blindly. Request and response bodies and headers are not collected, and observation never records read contents.
|
|
227
91
|
|
|
228
|
-
|
|
92
|
+
Tests and discovery do not call an AI service. Only the opt-in repair workflow, with `--allow-source-upload`, sends the selected source files and bounded observations to your configured OpenAI model; login state and command logs are not part of that upload. It sets `store: false`, which is not a promise of zero provider retention, so review your employer's policy and the provider's data terms first. A repair is a reviewable candidate, not a guaranteed fix.
|
|
229
93
|
|
|
230
94
|
## Scope
|
|
231
95
|
|
|
232
|
-
Recovery Probe
|
|
96
|
+
Recovery Probe checks request-level recovery deterministically and observes whether the screen reflects the reads it received. It does not simulate whole-device offline states, WebSocket interruptions, token expiry, browser sleep, slow networks or duplicate submissions.
|
|
233
97
|
|
|
234
|
-
|
|
98
|
+
Structural discovery compares healthy and faulted screens without a hand-written content selector, but it needs a selected read and a stable healthy baseline. No workflow infers your product's intended recovery policy.
|
|
235
99
|
|
|
236
|
-
The
|
|
100
|
+
The recovery, discovery and refresh workflows reload the renderer to establish controls. That clears in-memory caches, so those modes cannot prove account isolation. Observation keeps that state, but it does not infer identities or data ownership.
|
|
237
101
|
|
|
238
|
-
|
|
102
|
+
Attach only to authorized test sessions, keep debugging endpoints on loopback, and keep the target window visible and uncovered.
|
|
239
103
|
|
|
240
104
|
## Development
|
|
241
105
|
|
|
242
|
-
|
|
243
|
-
npm ci
|
|
244
|
-
npm test # fetch/core tests
|
|
245
|
-
npm run test:package # isolated tarball install + TypeScript + CLI checks
|
|
246
|
-
npx playwright install chromium
|
|
247
|
-
npm run test:browser # real-browser integration tests
|
|
248
|
-
npm run test:repair # safety, authenticated portal, before/after pipeline
|
|
249
|
-
# After installing the optional Electron test runtime:
|
|
250
|
-
npm run test:repair:electron
|
|
251
|
-
npm run demo:browser
|
|
252
|
-
```
|
|
253
|
-
|
|
254
|
-
CI runs package checks on Node.js 22 and 24, Chromium integration, and real Electron checks. Repair integration tests use a **controlled mock model response** but real apps, fault injection, source changes and reruns. They validate orchestration, not live AI diagnosis quality. See [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md).
|
|
106
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, checks and test suites, and [SECURITY.md](SECURITY.md) for reporting security issues. CI runs package checks on Node.js 22 and 24, Chromium integration, and real Electron checks.
|
|
255
107
|
|
|
256
108
|
## License
|
|
257
109
|
|
package/bin/ipc.mjs
CHANGED
|
@@ -3,14 +3,8 @@ import { readFile, mkdir, writeFile, mkdtemp } from 'node:fs/promises'
|
|
|
3
3
|
import { resolve, join } from 'node:path'
|
|
4
4
|
import { discoverIpc, discoveryConfig } from '../src/ipc-discover.mjs'
|
|
5
5
|
import { checkIpc } from '../src/ipc-check.mjs'
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
/[&<>"']/g,
|
|
9
|
-
(c) =>
|
|
10
|
-
({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[
|
|
11
|
-
c
|
|
12
|
-
],
|
|
13
|
-
)
|
|
6
|
+
import { renderTemplate } from '../src/report-template.mjs'
|
|
7
|
+
|
|
14
8
|
export async function ipcMain(args) {
|
|
15
9
|
if (!args.length || args.includes('--help')) {
|
|
16
10
|
console.log(`Recovery Probe IPC preview
|
|
@@ -71,7 +65,14 @@ No HTTP/Rust transport faults or AI calls. See docs/ipc.md for installation and
|
|
|
71
65
|
JSON.stringify(report, null, 2),
|
|
72
66
|
{ mode: 0o600 },
|
|
73
67
|
)
|
|
74
|
-
const html =
|
|
68
|
+
const html = renderTemplate('ipc', {
|
|
69
|
+
heading: report.ok
|
|
70
|
+
? 'Selected IPC experiments completed without findings requiring review'
|
|
71
|
+
: 'IPC recovery needs investigation',
|
|
72
|
+
limitation: report.limitation,
|
|
73
|
+
finalReset: report.finalReset,
|
|
74
|
+
evidence: JSON.stringify(report, null, 2),
|
|
75
|
+
})
|
|
75
76
|
await writeFile(join(directory, 'report.html'), html, { mode: 0o600 })
|
|
76
77
|
if (options.discover) {
|
|
77
78
|
await writeDiscoveryReport(directory, report, config)
|
package/bin/refresh.mjs
CHANGED
|
@@ -13,14 +13,8 @@ import {
|
|
|
13
13
|
httpRefreshAdapter,
|
|
14
14
|
ipcRefreshAdapter,
|
|
15
15
|
} from '../src/refresh-adapters.mjs'
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
/[&<>"']/g,
|
|
19
|
-
(c) =>
|
|
20
|
-
({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[
|
|
21
|
-
c
|
|
22
|
-
],
|
|
23
|
-
)
|
|
16
|
+
import { escapeHtml, renderTemplate } from '../src/report-template.mjs'
|
|
17
|
+
|
|
24
18
|
export async function refreshMain(args) {
|
|
25
19
|
if (args.includes('--help')) {
|
|
26
20
|
console.log(
|
|
@@ -136,7 +130,13 @@ export async function refreshMain(args) {
|
|
|
136
130
|
)
|
|
137
131
|
await writeFile(
|
|
138
132
|
resolve(directory, 'report.html'),
|
|
139
|
-
|
|
133
|
+
renderTemplate('refresh', {
|
|
134
|
+
limitation: report.limitation ?? report.error,
|
|
135
|
+
results: report.results
|
|
136
|
+
.map((r) => `<p>${escapeHtml(r.summary ?? r.code ?? r.stage)}</p>`)
|
|
137
|
+
.join(''),
|
|
138
|
+
evidence: JSON.stringify(report, null, 2),
|
|
139
|
+
}),
|
|
140
140
|
{ mode: 0o600 },
|
|
141
141
|
)
|
|
142
142
|
const summary = {
|
package/docs/api.md
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Library API
|
|
2
|
+
|
|
3
|
+
The library adds bounded faults to tests you already have. `recovery-probe` needs Playwright; `recovery-probe/fetch` has no runtime dependencies.
|
|
4
|
+
|
|
5
|
+
## Fault kinds
|
|
6
|
+
|
|
7
|
+
| Kind | Injected behavior |
|
|
8
|
+
| --- | --- |
|
|
9
|
+
| `http-error` | Fulfils the matched request with an HTTP error status; defaults to `503`. |
|
|
10
|
+
| `connection-failure` | Aborts the matched Playwright request with `connectionfailed`, or throws from the fetch adapter. |
|
|
11
|
+
| `invalid-json` | Returns status `200` with a deliberately malformed JSON body. |
|
|
12
|
+
|
|
13
|
+
Faults are bounded by `count` (default `1`, maximum `20`). The count is reserved synchronously, so concurrent matching requests cannot over-inject.
|
|
14
|
+
|
|
15
|
+
## `withFault(page, options, exercise)`
|
|
16
|
+
|
|
17
|
+
Installs one temporary Playwright route, runs `exercise`, verifies the requested fault count, and removes only that route handler. See the [README example](../README.md#add-a-recovery-check-to-a-playwright-test).
|
|
18
|
+
|
|
19
|
+
```ts
|
|
20
|
+
interface FaultOptions {
|
|
21
|
+
match: string | RegExp;
|
|
22
|
+
kind: 'http-error' | 'connection-failure' | 'invalid-json';
|
|
23
|
+
status?: number;
|
|
24
|
+
count?: number;
|
|
25
|
+
method?: string;
|
|
26
|
+
}
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
If the endpoint does not match, it throws `FAULT_NOT_TRIGGERED` instead of allowing the test to pass silently. If the fault is applied but the recovery assertion fails, it throws `RECOVERY_ASSERTION_FAILED` and retains the original assertion error as `cause`.
|
|
30
|
+
|
|
31
|
+
Register the fault after your own route mocks. The helper uses `route.fallback()` for requests it does not inject, allowing earlier handlers to continue processing them.
|
|
32
|
+
|
|
33
|
+
## `createFaultFetch(baseFetch, options)`
|
|
34
|
+
|
|
35
|
+
Creates an isolated fetch wrapper for one exact URL and method. It returns `{ fetch, summary, assertApplied }` and never patches global `fetch`. The URL must be absolute and matches exactly, including its query string. Every other request, including a relative one such as `fetch('/api/x')`, reaches the supplied fetch untouched.
|
|
36
|
+
|
|
37
|
+
```js
|
|
38
|
+
import { createFaultFetch } from 'recovery-probe/fetch';
|
|
39
|
+
|
|
40
|
+
const probe = createFaultFetch(fetch, {
|
|
41
|
+
url: 'http://127.0.0.1:3000/api/profile',
|
|
42
|
+
kind: 'connection-failure',
|
|
43
|
+
});
|
|
44
|
+
|
|
45
|
+
await exerciseApplication(probe.fetch);
|
|
46
|
+
probe.assertApplied();
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
`exerciseApplication` represents your own test adapter or application controller. Calling `assertApplied()` is essential: it proves that every requested fault was consumed.
|
|
50
|
+
|
|
51
|
+
## `probeRecovery(browser, config)`
|
|
52
|
+
|
|
53
|
+
Runs the baseline and the three default recovery checks used by the [JSON-configured runner](runner.md). For custom interactions or stronger assertions, prefer `withFault` inside your own test.
|
|
54
|
+
|
|
55
|
+
## When plain Playwright routes are enough
|
|
56
|
+
|
|
57
|
+
Playwright already provides the routing primitives used here. Recovery Probe adds bounded recipes, fault-occurrence verification, cleanup, consistent result codes, a fetch adapter, and a runnable broken-versus-fixed example. If a few direct `page.route()` calls are clearer for your test, use them; this package is most useful when teams want the recovery pattern to be repeatable.
|
package/docs/observe.md
CHANGED
|
@@ -108,7 +108,7 @@ Tracking ends at the next operator interval or capture termination, including in
|
|
|
108
108
|
|
|
109
109
|
## Actions, tracked rows and delayed UI updates
|
|
110
110
|
|
|
111
|
-
Available from **0.4.0-preview.1**. The current preview target is **0.4.0-preview.
|
|
111
|
+
Available from **0.4.0-preview.1**. The current preview target is **0.4.0-preview.4** on npm `next`; install it explicitly with `npm install --save-dev --save-exact recovery-probe@0.4.0-preview.4 playwright@1.62.1`. The development IPC hook must come from the same version.
|
|
112
112
|
|
|
113
113
|
These options are opt-in and extend the same fault-free capture. They answer "how long after the user's action did the screen show the right state, and did data arrive before it?"
|
|
114
114
|
|
package/docs/runner.md
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# JSON-configured runner
|
|
2
|
+
|
|
3
|
+
The original CLI runner suits pages that automatically load GET data, expose a visible Retry button after failure, and show a ready element after recovery. For other screens, use [guided setup](start.md), [discovery](desktop.md) or `withFault` in your own test ([API](api.md)).
|
|
4
|
+
|
|
5
|
+
```json
|
|
6
|
+
{
|
|
7
|
+
"name": "Profile recovery",
|
|
8
|
+
"url": "http://127.0.0.1:3000/profile",
|
|
9
|
+
"requestPattern": "**/api/profile",
|
|
10
|
+
"readySelector": "[data-testid=profile-ready]",
|
|
11
|
+
"retrySelector": "[data-testid=profile-retry]",
|
|
12
|
+
"timeoutMs": 3000
|
|
13
|
+
}
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npx recovery-probe --config recovery-probe.json --out artifacts/profile-recovery.json
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
The runner first checks the normal baseline in a fresh browser context. If that baseline fails, fault comparisons are skipped and reported as inconclusive. Each fault then receives its own fresh context with service workers blocked so Playwright can intercept the request.
|
|
21
|
+
|
|
22
|
+
The runner requires explicit selectors and endpoint matching. It does not discover an application's recovery policy.
|
|
23
|
+
|
|
24
|
+
## Exit codes
|
|
25
|
+
|
|
26
|
+
- `0`: every configured check passed, or the controlled demo observed its expected broken-versus-fixed result.
|
|
27
|
+
- `1`: a recovery check failed, was inconclusive, or was skipped.
|
|
28
|
+
- `2`: invalid arguments, configuration, or environment setup.
|
|
29
|
+
|
|
30
|
+
## Result codes
|
|
31
|
+
|
|
32
|
+
| Code | Meaning |
|
|
33
|
+
| --- | --- |
|
|
34
|
+
| `OK` | The configured assertion completed and the expected fault count was observed. |
|
|
35
|
+
| `RECOVERY_ASSERTION_FAILED` | The fault occurred, but the recovery assertion did not pass. |
|
|
36
|
+
| `FAULT_NOT_TRIGGERED` | The requested fault count was not observed. Check the route, method, cache, and service worker. |
|
|
37
|
+
| `BASELINE_FAILED` | The ordinary flow failed, so the runner could not make a useful fault comparison. |
|
|
38
|
+
| `INJECTION_ERROR` | Playwright could not apply or clean up the route reliably. |
|
|
39
|
+
|
|
40
|
+
A passing result proves the configured recovery behavior for that test. It is not a certification of overall application resilience.
|
package/package.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "recovery-probe",
|
|
3
|
-
"version": "0.4.0-preview.
|
|
3
|
+
"version": "0.4.0-preview.4",
|
|
4
4
|
"type": "module",
|
|
5
|
-
"description": "Test
|
|
5
|
+
"description": "Test whether web and Electron desktop apps show the right data.",
|
|
6
6
|
"license": "MIT",
|
|
7
7
|
"author": "Bojan Kovachki",
|
|
8
8
|
"homepage": "https://github.com/BojanKovachki/recovery-probe#readme",
|
|
@@ -58,7 +58,9 @@
|
|
|
58
58
|
"docs/ipc.md",
|
|
59
59
|
"docs/refresh.md",
|
|
60
60
|
"docs/observe.md",
|
|
61
|
-
"docs/ci.md"
|
|
61
|
+
"docs/ci.md",
|
|
62
|
+
"docs/api.md",
|
|
63
|
+
"docs/runner.md"
|
|
62
64
|
],
|
|
63
65
|
"sideEffects": false,
|
|
64
66
|
"engines": {
|
|
@@ -104,7 +106,10 @@
|
|
|
104
106
|
"fetch",
|
|
105
107
|
"electron",
|
|
106
108
|
"ipc",
|
|
107
|
-
"refresh"
|
|
109
|
+
"refresh",
|
|
110
|
+
"observation",
|
|
111
|
+
"ui-state",
|
|
112
|
+
"cache"
|
|
108
113
|
],
|
|
109
114
|
"scripts": {
|
|
110
115
|
"format": "prettier --write .",
|
|
@@ -124,7 +129,7 @@
|
|
|
124
129
|
"test:repair": "node --test test/repair.test.mjs test/repair.integration.test.mjs test/web.integration.test.mjs",
|
|
125
130
|
"test:repair:electron": "node --test test/repair.electron.test.mjs",
|
|
126
131
|
"test:guided": "node --test test/guided-start.test.mjs test/guided-start.integration.test.mjs",
|
|
127
|
-
"test:ipc": "node --test test/ipc-probe.test.mjs test/ipc-control.test.mjs test/ipc-observation.test.mjs test/visibility.test.mjs test/refresh.test.mjs test/observe.test.mjs test/observation-report.test.mjs test/observe-features.test.mjs",
|
|
132
|
+
"test:ipc": "node --test test/ipc-probe.test.mjs test/ipc-control.test.mjs test/ipc-observation.test.mjs test/visibility.test.mjs test/refresh.test.mjs test/observe.test.mjs test/observation-report.test.mjs test/report-template.test.mjs test/observe-features.test.mjs",
|
|
128
133
|
"test:ipc:electron": "node --test --test-concurrency=1 test/ipc.electron.test.mjs test/observe.electron.test.mjs test/observe-queue.electron.test.mjs",
|
|
129
134
|
"benchmark:observe": "node scripts/benchmark-observe.mjs",
|
|
130
135
|
"test:ci-example": "node examples/ci/verify.mjs"
|
package/src/desktop-report.mjs
CHANGED
|
@@ -1,29 +1,40 @@
|
|
|
1
1
|
import { mkdir, writeFile } from 'node:fs/promises'
|
|
2
2
|
import { resolve, join } from 'node:path'
|
|
3
|
+
import { escapeHtml, renderTemplate } from './report-template.mjs'
|
|
3
4
|
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
/[&<>"']/g,
|
|
7
|
-
(c) =>
|
|
8
|
-
({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[
|
|
9
|
-
c
|
|
10
|
-
],
|
|
11
|
-
)
|
|
12
|
-
export function desktopReportHtml(report) {
|
|
13
|
-
const rows = report.results
|
|
5
|
+
export function desktopReportHtml(report, target = 'desktop') {
|
|
6
|
+
const results = report.results
|
|
14
7
|
.map(
|
|
15
8
|
(row) =>
|
|
16
|
-
`<tr><td>${
|
|
9
|
+
`<tr><td>${escapeHtml(row.kind)}</td><td class="${escapeHtml(row.outcome)}">${escapeHtml(row.outcome.toUpperCase())}</td><td>${escapeHtml(row.code)}</td><td>${row.applied}</td><td>${row.successfulResponsesAfterFault}</td><td>${row.retryClicked ? 'Yes' : 'No'}</td></tr>`,
|
|
17
10
|
)
|
|
18
11
|
.join('')
|
|
19
|
-
const
|
|
12
|
+
const screenshots = report.results
|
|
20
13
|
.filter((row) => row.screenshot && /^[a-z0-9-]+\.png$/.test(row.screenshot))
|
|
21
14
|
.map(
|
|
22
15
|
(row) =>
|
|
23
|
-
`<figure><figcaption>${
|
|
16
|
+
`<figure><figcaption>${escapeHtml(row.kind)} — ${escapeHtml(row.code)}</figcaption><img alt="Renderer at the end of the recovery check" src="${escapeHtml(row.screenshot)}"></figure>`,
|
|
24
17
|
)
|
|
25
18
|
.join('')
|
|
26
|
-
return
|
|
19
|
+
return renderTemplate('desktop', {
|
|
20
|
+
target,
|
|
21
|
+
targetLabel: target.toUpperCase(),
|
|
22
|
+
heading: report.ok
|
|
23
|
+
? 'Selected recovery checks passed'
|
|
24
|
+
: 'Recovery needs investigation',
|
|
25
|
+
endpoint: report.endpoint,
|
|
26
|
+
recovery: report.recovery,
|
|
27
|
+
timeoutMs: report.timeoutMs,
|
|
28
|
+
generatedAt: report.generatedAt,
|
|
29
|
+
results,
|
|
30
|
+
finalReset: report.finalReset,
|
|
31
|
+
limitation: report.limitation,
|
|
32
|
+
reproduceWhere:
|
|
33
|
+
target === 'web'
|
|
34
|
+
? 'against the configured development URL'
|
|
35
|
+
: 'while your development app is on the same screen',
|
|
36
|
+
screenshots,
|
|
37
|
+
})
|
|
27
38
|
}
|
|
28
39
|
export async function writeDesktopReport(directory, report, config) {
|
|
29
40
|
const dir = resolve(directory)
|
|
@@ -43,23 +54,14 @@ export async function writeDesktopReport(directory, report, config) {
|
|
|
43
54
|
for (const [name, value] of Object.entries({
|
|
44
55
|
'report.json': JSON.stringify(report, null, 2) + '\n',
|
|
45
56
|
'scenario.json': JSON.stringify(config, null, 2) + '\n',
|
|
46
|
-
'report.html': desktopReportHtml(
|
|
57
|
+
'report.html': desktopReportHtml(
|
|
58
|
+
report,
|
|
59
|
+
config.target === 'web' ? 'web' : 'desktop',
|
|
60
|
+
),
|
|
47
61
|
'fix-brief.md': brief,
|
|
48
62
|
'reproduce.mjs': reproduction,
|
|
49
63
|
})) {
|
|
50
|
-
await writeFile(
|
|
51
|
-
join(dir, name),
|
|
52
|
-
config.target === 'web' && name === 'report.html'
|
|
53
|
-
? value
|
|
54
|
-
.replaceAll('desktop report', 'web report')
|
|
55
|
-
.replaceAll('DESKTOP PREVIEW', 'WEB PREVIEW')
|
|
56
|
-
.replace(
|
|
57
|
-
'while your development app is on the same screen',
|
|
58
|
-
'against the configured development URL',
|
|
59
|
-
)
|
|
60
|
-
: value,
|
|
61
|
-
{ mode: 0o600 },
|
|
62
|
-
)
|
|
64
|
+
await writeFile(join(dir, name), value, { mode: 0o600 })
|
|
63
65
|
}
|
|
64
66
|
return dir
|
|
65
67
|
}
|
package/src/discovery-report.mjs
CHANGED
|
@@ -1,13 +1,7 @@
|
|
|
1
1
|
import { writeFile } from 'node:fs/promises'
|
|
2
2
|
import { join } from 'node:path'
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
/[&<>"']/g,
|
|
6
|
-
(c) =>
|
|
7
|
-
({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[
|
|
8
|
-
c
|
|
9
|
-
],
|
|
10
|
-
)
|
|
3
|
+
import { escapeHtml, renderTemplate } from './report-template.mjs'
|
|
4
|
+
|
|
11
5
|
export async function writeDiscoveryReport(directory, report, config) {
|
|
12
6
|
await writeFile(join(directory, '.gitignore'), '*\n', { mode: 0o600 })
|
|
13
7
|
await writeFile(
|
|
@@ -20,10 +14,10 @@ export async function writeDiscoveryReport(directory, report, config) {
|
|
|
20
14
|
JSON.stringify(report, null, 2),
|
|
21
15
|
{ mode: 0o600 },
|
|
22
16
|
)
|
|
23
|
-
const
|
|
17
|
+
const experiments = report.findings
|
|
24
18
|
.map(
|
|
25
19
|
(f) =>
|
|
26
|
-
`<tr><td>${
|
|
20
|
+
`<tr><td>${escapeHtml(f.id)}</td><td>${escapeHtml(f.classification)}</td><td>${escapeHtml(f.kind)}</td><td>${escapeHtml(f.confidence ?? 'not-rated')}</td><td>${f.partial ? `${f.completedRuns}/${f.plannedRuns} (partial)` : f.confirmations}</td><td>${escapeHtml(f.observedInterpretation?.classification ?? '')}</td></tr>`,
|
|
27
21
|
)
|
|
28
22
|
.join('')
|
|
29
23
|
const summary = {
|
|
@@ -42,7 +36,12 @@ export async function writeDiscoveryReport(directory, report, config) {
|
|
|
42
36
|
}
|
|
43
37
|
await writeFile(
|
|
44
38
|
join(directory, 'report.html'),
|
|
45
|
-
|
|
39
|
+
renderTemplate('discovery', {
|
|
40
|
+
limitation: report.limitation,
|
|
41
|
+
experiments,
|
|
42
|
+
summary: JSON.stringify(summary, null, 2),
|
|
43
|
+
evidence: JSON.stringify(report, null, 2),
|
|
44
|
+
}),
|
|
46
45
|
{ mode: 0o600 },
|
|
47
46
|
)
|
|
48
47
|
}
|
package/src/fetch-fault.mjs
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
const kinds = new Set(['http-error', 'connection-failure', 'invalid-json'])
|
|
2
2
|
|
|
3
|
+
// A relative or malformed URL cannot be the absolute target, so it is never matched.
|
|
4
|
+
const absoluteHref = (value) => {
|
|
5
|
+
try {
|
|
6
|
+
return new URL(value).href
|
|
7
|
+
} catch {
|
|
8
|
+
return null
|
|
9
|
+
}
|
|
10
|
+
}
|
|
11
|
+
|
|
3
12
|
/** Fault-injected fetch for an exact, absolute URL. Each instance is isolated. */
|
|
4
13
|
export function createFaultFetch(baseFetch, options) {
|
|
5
14
|
if (typeof baseFetch !== 'function')
|
|
@@ -54,14 +63,13 @@ export function createFaultFetch(baseFetch, options) {
|
|
|
54
63
|
},
|
|
55
64
|
async fetch(input, init) {
|
|
56
65
|
const inputUrl = input instanceof Request ? input.url : input
|
|
57
|
-
const requestUrl = new URL(inputUrl)
|
|
58
66
|
const requestMethod = (
|
|
59
67
|
init?.method ?? (input instanceof Request ? input.method : 'GET')
|
|
60
68
|
).toUpperCase()
|
|
61
69
|
const signal =
|
|
62
70
|
init?.signal ?? (input instanceof Request ? input.signal : undefined)
|
|
63
71
|
if (
|
|
64
|
-
|
|
72
|
+
absoluteHref(inputUrl) !== target.href ||
|
|
65
73
|
requestMethod !== method ||
|
|
66
74
|
signal?.aborted
|
|
67
75
|
) {
|
package/src/ipc-probe.cjs
CHANGED
|
@@ -128,18 +128,23 @@ exports.installIpcProbe = function installIpcProbe(
|
|
|
128
128
|
}
|
|
129
129
|
const unwrapSend = () => {
|
|
130
130
|
if (!wrappedSend) return
|
|
131
|
-
const { contents, wrapper } = wrappedSend
|
|
132
|
-
//
|
|
131
|
+
const { contents, wrapper, previous } = wrappedSend
|
|
132
|
+
// Undo only our own change: restore an instance-level send that existed before the
|
|
133
|
+
// lease (for example an app's tracing wrapper), or remove ours to expose the prototype
|
|
134
|
+
// method again. A wrapper installed after ours stays; ours then only passes calls through.
|
|
133
135
|
if (
|
|
134
136
|
!contents.isDestroyed?.() &&
|
|
135
137
|
Object.hasOwn(contents, 'send') &&
|
|
136
138
|
contents.send === wrapper
|
|
137
|
-
)
|
|
138
|
-
|
|
139
|
+
) {
|
|
140
|
+
if (previous) Object.defineProperty(contents, 'send', previous)
|
|
141
|
+
else delete contents.send
|
|
142
|
+
}
|
|
139
143
|
wrappedSend = null
|
|
140
144
|
}
|
|
141
145
|
const wrapSend = (contents, current) => {
|
|
142
146
|
if (!allowedPushes.size || typeof contents?.send !== 'function') return
|
|
147
|
+
const previous = Object.getOwnPropertyDescriptor(contents, 'send')
|
|
143
148
|
const send = contents.send
|
|
144
149
|
const wrapper = function (channel, ...args) {
|
|
145
150
|
if (
|
|
@@ -172,7 +177,7 @@ exports.installIpcProbe = function installIpcProbe(
|
|
|
172
177
|
configurable: true,
|
|
173
178
|
writable: true,
|
|
174
179
|
})
|
|
175
|
-
wrappedSend = { contents, wrapper }
|
|
180
|
+
wrappedSend = { contents, wrapper, previous }
|
|
176
181
|
}
|
|
177
182
|
const disarm = () => {
|
|
178
183
|
clearTimeout(timer)
|
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { createRequire } from 'node:module'
|
|
2
|
+
import { escapeHtml, renderTemplate } from './report-template.mjs'
|
|
2
3
|
const { version } = createRequire(import.meta.url)('../package.json')
|
|
3
4
|
|
|
4
5
|
export function observationMetadata() {
|
|
@@ -306,16 +307,18 @@ export function comparisonDigest(report) {
|
|
|
306
307
|
)
|
|
307
308
|
}
|
|
308
309
|
|
|
309
|
-
const
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
)
|
|
310
|
+
const trackedRowHtml = (r) => {
|
|
311
|
+
const changes = r.transitions
|
|
312
|
+
.map(
|
|
313
|
+
(t) =>
|
|
314
|
+
`${escapeHtml(t.from)} → ${escapeHtml(t.to)} at ${seconds(t.afterMs)}–${seconds(t.byMs)} s`,
|
|
315
|
+
)
|
|
316
|
+
.join('<br>')
|
|
317
|
+
return `<tr><td>${escapeHtml(rowName(r.row))}</td><td>${seconds(r.firstSeen.fromMs)}–${seconds(r.firstSeen.toMs)} s as ${escapeHtml(r.firstSeen.state)}</td><td>${changes || 'none'}</td><td>${escapeHtml(r.lastState)}</td></tr>`
|
|
318
|
+
}
|
|
319
|
+
|
|
317
320
|
export function renderObservationReport(report) {
|
|
318
|
-
const
|
|
321
|
+
const intervals = report.intervals
|
|
319
322
|
.map(
|
|
320
323
|
(i) =>
|
|
321
324
|
`<tr><td>${i.index}</td><td>${i.reads?.calls ?? 'unknown'}</td><td>${i.reads?.successes ?? 'unknown'}</td><td>${i.reads?.errors ?? 'unknown'}</td><td>${i.firstContent ? `${i.firstContent.fromMs - i.startMs}–${i.firstContent.toMs - i.startMs} ms` : 'not sampled'}</td></tr>`,
|
|
@@ -325,27 +328,58 @@ export function renderObservationReport(report) {
|
|
|
325
328
|
const comparisons = (report.comparisons ?? [])
|
|
326
329
|
.map(
|
|
327
330
|
(c) =>
|
|
328
|
-
`<p>Interval ${
|
|
331
|
+
`<p>Interval ${escapeHtml(c.interval)}: ${escapeHtml(c.summary)} ${c.completions ? `(${c.completions.successes} successful, ${c.completions.errors} failed read completions)` : `(${c.reads.calls} selected reads started in this interval)`} ${escapeHtml(comparisonFollowUpSummary(c))}</p>`,
|
|
329
332
|
)
|
|
330
333
|
.join('')
|
|
331
|
-
const
|
|
334
|
+
const phaseRows = Object.entries(sampling?.phases ?? {})
|
|
332
335
|
.map(
|
|
333
336
|
([name, s]) =>
|
|
334
|
-
`<tr><td>${
|
|
337
|
+
`<tr><td>${escapeHtml(name)}</td><td>${s.meanMs}</td><td>${s.maxMs}</td></tr>`,
|
|
335
338
|
)
|
|
336
339
|
.join('')
|
|
337
340
|
const effects = report.actionEffects ?? []
|
|
338
341
|
const actionsHtml = effects.length
|
|
339
|
-
? `<h2>Actions</h2>${effects.map((e) => `<p>${
|
|
342
|
+
? `<h2>Actions</h2>${effects.map((e) => `<p>${escapeHtml(actionSummary(e))}</p>`).join('')}`
|
|
340
343
|
: ''
|
|
341
|
-
const
|
|
342
|
-
? `<h2>Tracked rows</h2><table><thead><tr><th>Row</th><th>First seen</th><th>State changes</th><th>Last state</th></tr></thead><tbody>${report.rows.map(
|
|
344
|
+
const trackedRows = report.rows?.length
|
|
345
|
+
? `<h2>Tracked rows</h2><table><thead><tr><th>Row</th><th>First seen</th><th>State changes</th><th>Last state</th></tr></thead><tbody>${report.rows.map(trackedRowHtml).join('')}</tbody></table><p>Row keys are hashed unless an alias was configured. States outside the configured allowlist are recorded as “other”.</p>`
|
|
343
346
|
: ''
|
|
344
347
|
const notes = (report.notes ?? [])
|
|
345
|
-
.map((n) => `<p>Note: ${
|
|
348
|
+
.map((n) => `<p>Note: ${escapeHtml(n.note)}</p>`)
|
|
346
349
|
.join('')
|
|
347
350
|
const operatorHtml = report.operator
|
|
348
|
-
? `<p>Scripted steps: ${
|
|
351
|
+
? `<p>Scripted steps: ${escapeHtml(report.operator.status)}${report.operator.error ? ` — ${escapeHtml(report.operator.error)}` : ''}.</p>`
|
|
352
|
+
: ''
|
|
353
|
+
const samplingHtml = sampling
|
|
354
|
+
? `<p>Configured delay: ${sampling.requestedDelayMs} ms. Completed samples: ${sampling.completedSamples}. Mean sample duration: ${sampling.sampleDuration.meanMs ?? 'unavailable'} ms. Mean start-to-start cadence: ${sampling.startToStart.meanMs ?? 'unavailable'} ms (maximum ${sampling.startToStart.maxMs ?? 'unavailable'} ms).</p><p>${escapeHtml(sampling.note)}</p>`
|
|
349
355
|
: ''
|
|
350
|
-
|
|
356
|
+
const phases = phaseRows
|
|
357
|
+
? `<table><thead><tr><th>Measured phase</th><th>Mean ms</th><th>Maximum ms</th></tr></thead><tbody>${phaseRows}</tbody></table><p>Renderer read time is included in the UI round trip, not additional to it.</p>`
|
|
358
|
+
: ''
|
|
359
|
+
const findings = report.findings
|
|
360
|
+
.map(
|
|
361
|
+
(f) =>
|
|
362
|
+
`<p>Interval ${f.interval}: ${escapeHtml(observationSummary(f))} <strong>${escapeHtml(f.code)}</strong> — ${escapeHtml(f.basis)}</p>`,
|
|
363
|
+
)
|
|
364
|
+
.join('')
|
|
365
|
+
return renderTemplate('observation', {
|
|
366
|
+
status: report.status,
|
|
367
|
+
version: report.metadata?.recoveryProbeVersion ?? 'unknown',
|
|
368
|
+
node: report.metadata?.runtime?.node ?? 'unknown',
|
|
369
|
+
browser: report.metadata?.browserVersion ?? 'unknown',
|
|
370
|
+
ipcHook: report.metadata?.ipcHook?.version ?? 'unknown / not applicable',
|
|
371
|
+
sampling: samplingHtml,
|
|
372
|
+
detail: report.detail ?? 'region (legacy)',
|
|
373
|
+
phases,
|
|
374
|
+
limitation: report.limitation ?? report.error,
|
|
375
|
+
intervals,
|
|
376
|
+
findings,
|
|
377
|
+
operator: operatorHtml,
|
|
378
|
+
actions: actionsHtml,
|
|
379
|
+
trackedRows,
|
|
380
|
+
notes,
|
|
381
|
+
comparisons,
|
|
382
|
+
cleanup: report.cleanup ?? 'unknown',
|
|
383
|
+
evidence: JSON.stringify(report, null, 2),
|
|
384
|
+
})
|
|
351
385
|
}
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
import { readFileSync } from 'node:fs'
|
|
2
|
+
|
|
3
|
+
const entities = {
|
|
4
|
+
'&': '&',
|
|
5
|
+
'<': '<',
|
|
6
|
+
'>': '>',
|
|
7
|
+
'"': '"',
|
|
8
|
+
"'": ''',
|
|
9
|
+
}
|
|
10
|
+
|
|
11
|
+
/** Escapes text for HTML element content or a quoted attribute value. */
|
|
12
|
+
export const escapeHtml = (value) =>
|
|
13
|
+
String(value ?? '').replace(/[&<>"']/g, (c) => entities[c])
|
|
14
|
+
|
|
15
|
+
/**
|
|
16
|
+
* Fills `report-templates/<name>.html`. `{{slot}}` takes text and escapes it;
|
|
17
|
+
* `{{{slot}}}` takes markup the caller already built with escaped values.
|
|
18
|
+
*/
|
|
19
|
+
export function renderTemplate(name, slots) {
|
|
20
|
+
const template = readFileSync(
|
|
21
|
+
new URL(`./report-templates/${name}.html`, import.meta.url),
|
|
22
|
+
'utf8',
|
|
23
|
+
)
|
|
24
|
+
return template.replace(
|
|
25
|
+
/\{\{\{(\w+)\}\}\}|\{\{(\w+)\}\}/g,
|
|
26
|
+
(_, markupSlot, textSlot) => {
|
|
27
|
+
const slot = markupSlot ?? textSlot
|
|
28
|
+
if (!Object.hasOwn(slots, slot))
|
|
29
|
+
throw new Error(`Report template ${name} needs a value for ${slot}`)
|
|
30
|
+
return markupSlot ? String(slots[slot]) : escapeHtml(slots[slot])
|
|
31
|
+
},
|
|
32
|
+
)
|
|
33
|
+
}
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
<!doctype html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="utf-8" />
|
|
5
|
+
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
6
|
+
<title>Recovery Probe — {{target}} report</title>
|
|
7
|
+
<style>
|
|
8
|
+
body {
|
|
9
|
+
font: 16px/1.6 system-ui, sans-serif;
|
|
10
|
+
background: #f6f7f9;
|
|
11
|
+
color: #17212f;
|
|
12
|
+
margin: 0;
|
|
13
|
+
padding: 40px;
|
|
14
|
+
}
|
|
15
|
+
main {
|
|
16
|
+
max-width: 1100px;
|
|
17
|
+
margin: auto;
|
|
18
|
+
}
|
|
19
|
+
h1 {
|
|
20
|
+
font-size: 32px;
|
|
21
|
+
letter-spacing: -1px;
|
|
22
|
+
}
|
|
23
|
+
code {
|
|
24
|
+
overflow-wrap: anywhere;
|
|
25
|
+
}
|
|
26
|
+
table {
|
|
27
|
+
width: 100%;
|
|
28
|
+
border-collapse: collapse;
|
|
29
|
+
background: white;
|
|
30
|
+
font-size: 14px;
|
|
31
|
+
}
|
|
32
|
+
th,
|
|
33
|
+
td {
|
|
34
|
+
text-align: left;
|
|
35
|
+
border-bottom: 1px solid #ddd;
|
|
36
|
+
padding: 12px;
|
|
37
|
+
}
|
|
38
|
+
section {
|
|
39
|
+
background: white;
|
|
40
|
+
padding: 20px;
|
|
41
|
+
margin: 24px 0;
|
|
42
|
+
border-radius: 10px;
|
|
43
|
+
}
|
|
44
|
+
.pass {
|
|
45
|
+
color: #087443;
|
|
46
|
+
}
|
|
47
|
+
.fail {
|
|
48
|
+
color: #b3261e;
|
|
49
|
+
}
|
|
50
|
+
.inconclusive,
|
|
51
|
+
.skipped {
|
|
52
|
+
color: #755300;
|
|
53
|
+
}
|
|
54
|
+
img {
|
|
55
|
+
max-width: 100%;
|
|
56
|
+
border: 1px solid #ddd;
|
|
57
|
+
}
|
|
58
|
+
figcaption {
|
|
59
|
+
font-weight: 600;
|
|
60
|
+
}
|
|
61
|
+
small {
|
|
62
|
+
color: #536174;
|
|
63
|
+
}
|
|
64
|
+
.scroll {
|
|
65
|
+
overflow: auto;
|
|
66
|
+
}
|
|
67
|
+
</style>
|
|
68
|
+
</head>
|
|
69
|
+
<body>
|
|
70
|
+
<main>
|
|
71
|
+
<p>RECOVERY PROBE · {{targetLabel}} PREVIEW</p>
|
|
72
|
+
<h1>{{heading}}</h1>
|
|
73
|
+
<p><code>GET {{endpoint}}</code></p>
|
|
74
|
+
<p>
|
|
75
|
+
Recovery: <strong>{{recovery}}</strong> · Deadline: {{timeoutMs}} ms ·
|
|
76
|
+
{{generatedAt}}
|
|
77
|
+
</p>
|
|
78
|
+
<div class="scroll">
|
|
79
|
+
<table>
|
|
80
|
+
<thead>
|
|
81
|
+
<tr>
|
|
82
|
+
<th>Injected fault</th>
|
|
83
|
+
<th>Outcome</th>
|
|
84
|
+
<th>Result</th>
|
|
85
|
+
<th>Injected</th>
|
|
86
|
+
<th>Later successful responses</th>
|
|
87
|
+
<th>Retry clicked</th>
|
|
88
|
+
</tr>
|
|
89
|
+
</thead>
|
|
90
|
+
<tbody>
|
|
91
|
+
{{{results}}}
|
|
92
|
+
</tbody>
|
|
93
|
+
</table>
|
|
94
|
+
</div>
|
|
95
|
+
<section>
|
|
96
|
+
<h2>How to read this</h2>
|
|
97
|
+
<p>
|
|
98
|
+
<strong>PASS:</strong> the fault occurred, the configured action
|
|
99
|
+
completed, a later matching request succeeded, and the selected
|
|
100
|
+
healthy UI marker appeared. <strong>FAIL:</strong> after the fault
|
|
101
|
+
and action, recovery was not observed by the deadline. Investigate
|
|
102
|
+
the app and the chosen assertion. <strong>INCONCLUSIVE:</strong>
|
|
103
|
+
setup, baseline, interception or the recovery action could not be
|
|
104
|
+
verified; this is not a confirmed app bug.
|
|
105
|
+
</p>
|
|
106
|
+
<p>
|
|
107
|
+
Final reload reset: <strong>{{finalReset}}</strong>. A reload that
|
|
108
|
+
rescues a stuck page does not turn the original failed recovery check
|
|
109
|
+
into a pass.
|
|
110
|
+
</p>
|
|
111
|
+
<p>{{limitation}}</p>
|
|
112
|
+
</section>
|
|
113
|
+
<section>
|
|
114
|
+
<h2>Reproduce and investigate</h2>
|
|
115
|
+
<p>
|
|
116
|
+
From this output directory, run <code>node reproduce.mjs</code>
|
|
117
|
+
{{reproduceWhere}}. See <code>scenario.json</code> and
|
|
118
|
+
<code>fix-brief.md</code>.
|
|
119
|
+
</p>
|
|
120
|
+
<p>
|
|
121
|
+
No source patch was applied. No data was uploaded. Review local
|
|
122
|
+
artifacts before sharing them.
|
|
123
|
+
</p>
|
|
124
|
+
</section>
|
|
125
|
+
{{{screenshots}}}
|
|
126
|
+
<small>
|
|
127
|
+
HTTP status success and a visible marker do not certify response
|
|
128
|
+
semantics. Choose a marker that only appears when this endpoint's data
|
|
129
|
+
has actually loaded; configure readyText/busySelector for a stronger
|
|
130
|
+
assertion.
|
|
131
|
+
</small>
|
|
132
|
+
</main>
|
|
133
|
+
</body>
|
|
134
|
+
</html>
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
<!doctype html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="utf-8" />
|
|
5
|
+
<title>Recovery Probe discovery</title>
|
|
6
|
+
<style>
|
|
7
|
+
body {
|
|
8
|
+
font: 16px system-ui;
|
|
9
|
+
max-width: 1100px;
|
|
10
|
+
margin: 40px auto;
|
|
11
|
+
padding: 20px;
|
|
12
|
+
}
|
|
13
|
+
td,
|
|
14
|
+
th {
|
|
15
|
+
padding: 10px;
|
|
16
|
+
border-bottom: 1px solid #ddd;
|
|
17
|
+
text-align: left;
|
|
18
|
+
}
|
|
19
|
+
pre {
|
|
20
|
+
white-space: pre-wrap;
|
|
21
|
+
overflow-wrap: anywhere;
|
|
22
|
+
}
|
|
23
|
+
</style>
|
|
24
|
+
</head>
|
|
25
|
+
<body>
|
|
26
|
+
<h1>Recovery Probe discovery</h1>
|
|
27
|
+
<p>{{limitation}}</p>
|
|
28
|
+
<table>
|
|
29
|
+
<tr>
|
|
30
|
+
<th>Experiment</th>
|
|
31
|
+
<th>Interpretation</th>
|
|
32
|
+
<th>Kind</th>
|
|
33
|
+
<th>Confidence</th>
|
|
34
|
+
<th>Runs</th>
|
|
35
|
+
<th>Partial observation</th>
|
|
36
|
+
</tr>
|
|
37
|
+
{{{experiments}}}
|
|
38
|
+
</table>
|
|
39
|
+
<h2>Run summary</h2>
|
|
40
|
+
<pre>{{summary}}</pre>
|
|
41
|
+
<details>
|
|
42
|
+
<summary>Observations and bounded timelines</summary>
|
|
43
|
+
<pre>{{evidence}}</pre>
|
|
44
|
+
</details>
|
|
45
|
+
</body>
|
|
46
|
+
</html>
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
<!doctype html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="utf-8" />
|
|
5
|
+
<title>Recovery Probe IPC report</title>
|
|
6
|
+
<style>
|
|
7
|
+
body {
|
|
8
|
+
max-width: 1000px;
|
|
9
|
+
margin: 40px auto;
|
|
10
|
+
font: 16px system-ui;
|
|
11
|
+
padding: 20px;
|
|
12
|
+
}
|
|
13
|
+
pre {
|
|
14
|
+
white-space: pre-wrap;
|
|
15
|
+
overflow-wrap: anywhere;
|
|
16
|
+
background: #f4f5f7;
|
|
17
|
+
padding: 20px;
|
|
18
|
+
}
|
|
19
|
+
</style>
|
|
20
|
+
</head>
|
|
21
|
+
<body>
|
|
22
|
+
<h1>{{heading}}</h1>
|
|
23
|
+
<p>{{limitation}}</p>
|
|
24
|
+
<p>
|
|
25
|
+
Final reload: {{finalReset}}. Cleanup cannot turn an earlier failure into
|
|
26
|
+
a pass.
|
|
27
|
+
</p>
|
|
28
|
+
<pre>{{evidence}}</pre>
|
|
29
|
+
</body>
|
|
30
|
+
</html>
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
<!doctype html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="utf-8" />
|
|
5
|
+
<title>Recovery Probe observation</title>
|
|
6
|
+
<style>
|
|
7
|
+
body {
|
|
8
|
+
font: 16px system-ui;
|
|
9
|
+
max-width: 1000px;
|
|
10
|
+
margin: 40px auto;
|
|
11
|
+
padding: 20px;
|
|
12
|
+
}
|
|
13
|
+
td,
|
|
14
|
+
th {
|
|
15
|
+
text-align: left;
|
|
16
|
+
padding: 8px;
|
|
17
|
+
}
|
|
18
|
+
pre {
|
|
19
|
+
white-space: pre-wrap;
|
|
20
|
+
overflow-wrap: anywhere;
|
|
21
|
+
}
|
|
22
|
+
</style>
|
|
23
|
+
</head>
|
|
24
|
+
<body>
|
|
25
|
+
<h1>Fault-free observation</h1>
|
|
26
|
+
<p>
|
|
27
|
+
Capture: {{status}}. This is not a recovery or account-isolation verdict.
|
|
28
|
+
</p>
|
|
29
|
+
<p>
|
|
30
|
+
Recovery Probe {{version}} · Node {{node}} · Browser {{browser}} · IPC
|
|
31
|
+
hook {{ipcHook}}
|
|
32
|
+
</p>
|
|
33
|
+
{{{sampling}}}
|
|
34
|
+
<p>
|
|
35
|
+
Sampling detail: {{detail}}. Marker counts describe the configured
|
|
36
|
+
content selector; generic region item counts are heuristics.
|
|
37
|
+
</p>
|
|
38
|
+
{{{phases}}}
|
|
39
|
+
<p>{{limitation}}</p>
|
|
40
|
+
<table>
|
|
41
|
+
<thead>
|
|
42
|
+
<tr>
|
|
43
|
+
<th>Interval</th>
|
|
44
|
+
<th>Reads started</th>
|
|
45
|
+
<th>Successes</th>
|
|
46
|
+
<th>Errors</th>
|
|
47
|
+
<th>First content sample bounds</th>
|
|
48
|
+
</tr>
|
|
49
|
+
</thead>
|
|
50
|
+
<tbody>
|
|
51
|
+
{{{intervals}}}
|
|
52
|
+
</tbody>
|
|
53
|
+
</table>
|
|
54
|
+
{{{findings}}}
|
|
55
|
+
{{{operator}}}
|
|
56
|
+
{{{actions}}}
|
|
57
|
+
{{{trackedRows}}}
|
|
58
|
+
{{{notes}}}
|
|
59
|
+
{{{comparisons}}}
|
|
60
|
+
<p>
|
|
61
|
+
Comparisons concern sampled marker text and count, not ownership or
|
|
62
|
+
freshness. A completion does not guarantee the UI has consumed its
|
|
63
|
+
result.
|
|
64
|
+
</p>
|
|
65
|
+
<p>
|
|
66
|
+
Cleanup: {{cleanup}}. The observation snapshot is from before cleanup;
|
|
67
|
+
cleanupEvidence records the separate cleanup check when available.
|
|
68
|
+
</p>
|
|
69
|
+
<details>
|
|
70
|
+
<summary>Evidence and coverage</summary>
|
|
71
|
+
<pre>{{evidence}}</pre>
|
|
72
|
+
</details>
|
|
73
|
+
</body>
|
|
74
|
+
</html>
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
<!doctype html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="utf-8" />
|
|
5
|
+
<title>Recovery Probe refresh</title>
|
|
6
|
+
<style>
|
|
7
|
+
body {
|
|
8
|
+
font: 16px system-ui;
|
|
9
|
+
max-width: 1000px;
|
|
10
|
+
margin: 40px auto;
|
|
11
|
+
padding: 20px;
|
|
12
|
+
}
|
|
13
|
+
pre {
|
|
14
|
+
white-space: pre-wrap;
|
|
15
|
+
overflow-wrap: anywhere;
|
|
16
|
+
}
|
|
17
|
+
</style>
|
|
18
|
+
</head>
|
|
19
|
+
<body>
|
|
20
|
+
<h1>Refresh experiment</h1>
|
|
21
|
+
<p>{{limitation}}</p>
|
|
22
|
+
{{{results}}}
|
|
23
|
+
<details>
|
|
24
|
+
<summary>Evidence</summary>
|
|
25
|
+
<pre>{{evidence}}</pre>
|
|
26
|
+
</details>
|
|
27
|
+
</body>
|
|
28
|
+
</html>
|