@alwaysmeticulous/debug-workspace 2.304.0 → 2.306.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/__tests__/detect-snapshot-assets.spec.js +1 -4
- package/dist/__tests__/detect-snapshot-assets.spec.js.map +1 -1
- package/dist/__tests__/extract-screenshot-dom-files.spec.js +1 -4
- package/dist/__tests__/extract-screenshot-dom-files.spec.js.map +1 -1
- package/dist/__tests__/fetch-dom-diffs.spec.js +1 -4
- package/dist/__tests__/fetch-dom-diffs.spec.js.map +1 -1
- package/dist/__tests__/redact-cookies-and-headers.spec.js +1 -4
- package/dist/__tests__/redact-cookies-and-headers.spec.js.map +1 -1
- package/dist/__tests__/redact-log-credentials.spec.js +1 -4
- package/dist/__tests__/redact-log-credentials.spec.js.map +1 -1
- package/dist/debug-constants.js +1 -4
- package/dist/debug-constants.js.map +1 -1
- package/dist/debug.types.js +1 -4
- package/dist/debug.types.js.map +1 -1
- package/dist/download-debug-data.js +1 -4
- package/dist/download-debug-data.js.map +1 -1
- package/dist/extract-screenshot-dom-files.js +1 -4
- package/dist/extract-screenshot-dom-files.js.map +1 -1
- package/dist/fetch-dom-diffs.js +1 -4
- package/dist/fetch-dom-diffs.js.map +1 -1
- package/dist/generate-debug-derived-files.js +1 -4
- package/dist/generate-debug-derived-files.js.map +1 -1
- package/dist/generate-debug-workspace.js +1 -4
- package/dist/generate-debug-workspace.js.map +1 -1
- package/dist/index.js +1 -4
- package/dist/index.js.map +1 -1
- package/dist/pipeline.js +1 -4
- package/dist/pipeline.js.map +1 -1
- package/dist/replay-walk.js +1 -4
- package/dist/replay-walk.js.map +1 -1
- package/dist/resolve-debug-context.js +1 -4
- package/dist/resolve-debug-context.js.map +1 -1
- package/dist/screenshot-identifier.js +1 -4
- package/dist/screenshot-identifier.js.map +1 -1
- package/dist/templates/CLAUDE.md +1 -6
- package/dist/templates/templates/CLAUDE.md +348 -0
- package/dist/templates/templates/agents/log-diff-analyzer.md +66 -0
- package/dist/templates/templates/agents/planner.md +67 -0
- package/dist/templates/templates/agents/pr-analyzer.md +57 -0
- package/dist/templates/templates/agents/summarizer.md +75 -0
- package/dist/templates/templates/settings.json +57 -0
- package/dist/templates/templates/skills/debugging-diffs/SKILL.md +61 -0
- package/dist/templates/templates/skills/debugging-flakes/SKILL.md +52 -0
- package/dist/templates/templates/skills/debugging-network/SKILL.md +45 -0
- package/dist/templates/templates/skills/debugging-sessions/SKILL.md +47 -0
- package/dist/templates/templates/skills/debugging-timelines/SKILL.md +51 -0
- package/dist/templates/templates/skills/pr-analysis/SKILL.md +20 -0
- package/package.json +3 -3
|
@@ -0,0 +1,348 @@
|
|
|
1
|
+
# Meticulous Debug Workspace
|
|
2
|
+
|
|
3
|
+
## What This Is
|
|
4
|
+
|
|
5
|
+
You are in a debugging workspace for the Meticulous automated UI testing platform.
|
|
6
|
+
You are investigating a replay issue (flaky behavior, unexpected diffs, or replay failures).
|
|
7
|
+
|
|
8
|
+
`debug-data/context.json` is the index for this workspace. Read it first; it lists all IDs, paths,
|
|
9
|
+
metadata, and what data is available.
|
|
10
|
+
|
|
11
|
+
## How Meticulous Works
|
|
12
|
+
|
|
13
|
+
Meticulous records user sessions by injecting a JavaScript recorder snippet into your
|
|
14
|
+
application. These sessions capture user activity and network requests. When you make
|
|
15
|
+
a commit, Meticulous triggers a test run that replays selected sessions against the new code,
|
|
16
|
+
taking screenshots at key moments. If there is a base test run to compare against, screenshot
|
|
17
|
+
diffs are computed and surfaced to the developer.
|
|
18
|
+
|
|
19
|
+
- A **replay** is a single session being replayed against a version of your app.
|
|
20
|
+
- A **test run** is a collection of replays triggered by a commit.
|
|
21
|
+
- A **replay diff** compares a head replay (new code) against a base replay (old code) and
|
|
22
|
+
contains the screenshot diff results.
|
|
23
|
+
- A **session** is the original user recording that gets replayed.
|
|
24
|
+
|
|
25
|
+
## Workspace Layout
|
|
26
|
+
|
|
27
|
+
The workspace root contains the debug workspace files. All downloaded debug data lives under
|
|
28
|
+
the `debug-data/` subdirectory.
|
|
29
|
+
|
|
30
|
+
- **`debug-data/context.json`** -- Index of IDs, paths, metadata, and what data is available
|
|
31
|
+
in this workspace. Read this first.
|
|
32
|
+
- **`debug-data/`** -- All downloaded replay data, session recordings, diffs, and
|
|
33
|
+
pre-computed analysis artifacts.
|
|
34
|
+
<!-- if-local-cli -->
|
|
35
|
+
- **`project-repo/`** -- (Optional) Your codebase checked out at the relevant commit.
|
|
36
|
+
Only present if the command was run from within a git repository.
|
|
37
|
+
<!-- end-if-local-cli -->
|
|
38
|
+
|
|
39
|
+
## debug-data/ Contents
|
|
40
|
+
|
|
41
|
+
Data falls into three categories: per-replay files (always present), diff files (only when
|
|
42
|
+
comparing replays), and other data.
|
|
43
|
+
|
|
44
|
+
### Per-Replay Files (always available)
|
|
45
|
+
|
|
46
|
+
Replay data is organized into `head/` and `base/` subdirectories under
|
|
47
|
+
`debug-data/replays/`. All files are searchable and can be found via glob/search.
|
|
48
|
+
|
|
49
|
+
Each replay directory (`debug-data/replays/<role>/<replayId>/`) contains:
|
|
50
|
+
|
|
51
|
+
- `logs.deterministic.txt` -- Deterministic logs with non-deterministic data stripped. Best for
|
|
52
|
+
diffing between replays. Can be very large (check `fileMetadata` in `context.json` for sizes).
|
|
53
|
+
Credential-looking values (`Cookie`/`Authorization`/etc. header dumps, `Bearer`/`Basic` tokens)
|
|
54
|
+
are always redacted, even in this "raw" file; other noise (tunnel URLs, S3 tokens, PostHog
|
|
55
|
+
payloads) is left as-is here and only stripped in the filtered copy below.
|
|
56
|
+
- `logs.deterministic.filtered.txt` -- **Start here for single-replay investigation.**
|
|
57
|
+
Noise-stripped version of the deterministic logs: tunnel URLs, S3 tokens, PostHog payloads,
|
|
58
|
+
build hashes, and other non-deterministic patterns are replaced with placeholders. Prefer this
|
|
59
|
+
over the raw version unless you need unmodified output.
|
|
60
|
+
- `logs.concise.txt` -- Full logs with both virtual and real timestamps, and trace IDs.
|
|
61
|
+
Credential-looking values are redacted the same way as in `logs.deterministic.txt`.
|
|
62
|
+
- `timeline.json` -- Detailed timeline of all replay events (user interactions, network requests,
|
|
63
|
+
DOM mutations, etc.). Can be 1-2MB; prefer `debug-data/events-index/` or
|
|
64
|
+
`debug-data/timeline-summaries/` for compact overviews. Cookie/header values on network request
|
|
65
|
+
entries are redacted.
|
|
66
|
+
- `timeline.ndjson` -- Same data as `timeline.json` but one JSON object per line (NDJSON format).
|
|
67
|
+
Greppable with standard tools: `grep '"screenshot"' timeline.ndjson` to find screenshots,
|
|
68
|
+
`grep '"pollyReplay"' timeline.ndjson` for network stubs.
|
|
69
|
+
- `timeline-stats.json` -- Aggregated statistics about timeline events.
|
|
70
|
+
- `metadata.json` -- Replay configuration, parameters, and environment info.
|
|
71
|
+
- `launchBrowserAndReplayParams.json` -- The exact parameters used to launch the replay.
|
|
72
|
+
- `stack-traces.ndjson` -- JavaScript stack traces captured during replay (if any errors
|
|
73
|
+
occurred), one JSON object per line.
|
|
74
|
+
- `accuracy.json` -- Replay accuracy assessment comparing to expected behavior.
|
|
75
|
+
- `cookies.json` -- Cookies captured at session record time. Rarely needed; useful only when
|
|
76
|
+
authentication or cookie-driven behavior is part of the investigation.
|
|
77
|
+
- `mapped-coverage.json` -- JS code coverage for the whole replay, mapped back to source files.
|
|
78
|
+
Cross-reference with `pr-diff.txt` to check whether changed code actually executed during the replay.
|
|
79
|
+
- `mapped-per-screenshot-js-coverage/<screenshotId>.json` -- Same coverage broken down per
|
|
80
|
+
screenshot. Use this to localize which code ran around a specific diff.
|
|
81
|
+
<!-- if-snapshot-assets -->
|
|
82
|
+
- `snapshotted-assets/` -- Static assets (JS/CSS) that were captured and used during replay.
|
|
83
|
+
<!-- end-if-snapshot-assets -->
|
|
84
|
+
- `screenshots/<name>.metadata.json` -- Per-screenshot metadata, including the captured `before.dom`
|
|
85
|
+
(full HTML at screenshot time).
|
|
86
|
+
- `screenshots/<name>.html` -- The `before.dom` extracted to a standalone HTML file. Prefer reading
|
|
87
|
+
this over grepping the metadata JSON. The one-line `<!-- screenshot=... url=... vt=... -->`
|
|
88
|
+
header at the top is for humans only and is not used when computing DOM diffs.
|
|
89
|
+
|
|
90
|
+
Per-replay generated summaries:
|
|
91
|
+
|
|
92
|
+
- `debug-data/events-index/<role>-<replayId>.txt` -- **Use this instead of raw timeline.json.**
|
|
93
|
+
One line per timeline event with index, virtual time, kind, and key data fields. Fully greppable:
|
|
94
|
+
`grep 'kind=screenshot' events-index/head-abc.txt` to find screenshots,
|
|
95
|
+
`grep 'kind=pollyReplay' events-index/head-abc.txt` for network stubs,
|
|
96
|
+
`grep 'api/v9/users' events-index/head-abc.txt` for specific API calls.
|
|
97
|
+
- `debug-data/network-log/<role>-<replayId>.txt` -- Compact network request log with one line
|
|
98
|
+
per request: method, URL, status, and match result. Grep for specific endpoints.
|
|
99
|
+
- `debug-data/vt-progression/<role>-<replayId>.txt` -- One virtual time value per line, extracted
|
|
100
|
+
from `logs.ndjson`. Compare head vs base with `diff`. Use this instead of parsing `logs.ndjson`
|
|
101
|
+
for virtual time progression.
|
|
102
|
+
- `debug-data/logs-index/<role>-<replayId>.txt` -- **Use this instead of raw logs.ndjson.**
|
|
103
|
+
One line per log entry with index, current virtual time, source, type, and truncated message.
|
|
104
|
+
Virtual-time-change entries appear as `[virtual-time-change -> <value>]`. Fully greppable:
|
|
105
|
+
`grep 'source=application' logs-index/head-abc.txt` to filter by source,
|
|
106
|
+
`grep -i 'error' logs-index/head-abc.txt` for keyword search,
|
|
107
|
+
`grep 'vt=7648' logs-index/head-abc.txt` for events at a specific virtual time.
|
|
108
|
+
- `debug-data/screenshot-timeline-context/<role>-<replayId>-<screenshotId>.txt` -- Events
|
|
109
|
+
surrounding each screenshot (30 before, 10 after) from the timeline. The screenshot line is
|
|
110
|
+
marked with `>>>`. Use these to understand what happened right before a screenshot.
|
|
111
|
+
- `debug-data/timeline-summaries/<role>-<replayId>.txt` -- Compact summary of each replay's
|
|
112
|
+
timeline: total entries, virtual time range, screenshot timestamps, event kind breakdown.
|
|
113
|
+
<!-- if-snapshot-assets -->
|
|
114
|
+
- `debug-data/formatted-assets/<role>/<replayId>/` -- Pretty-printed JS/CSS from
|
|
115
|
+
`snapshotted-assets/`. Use these instead of the originals.
|
|
116
|
+
<!-- end-if-snapshot-assets -->
|
|
117
|
+
|
|
118
|
+
### Diff Files (only when comparing replays)
|
|
119
|
+
|
|
120
|
+
These files are only generated when comparing replays.
|
|
121
|
+
|
|
122
|
+
- `debug-data/diffs/<id>.json` -- Full diff data including replay metadata, test run config,
|
|
123
|
+
and screenshot results. Can be very large (20K+ tokens). Only read this if you need the full context.
|
|
124
|
+
- `debug-data/diffs/<id>.summary.json` -- **Start here.** Compact summary with just the screenshot
|
|
125
|
+
diff results: which screenshots differ, mismatch pixel counts, mismatch percentages, and changed
|
|
126
|
+
section class names.
|
|
127
|
+
- `debug-data/log-diffs/<id>.diff` -- Raw unified diff of `logs.deterministic.txt` between head and base.
|
|
128
|
+
- `debug-data/log-diffs/<id>.filtered.diff` -- **Start here for diff investigation.** Noise-stripped
|
|
129
|
+
version with tunnel URLs, S3 tokens, PostHog payloads removed. Hunks that only differ in
|
|
130
|
+
noise are removed entirely.
|
|
131
|
+
- `debug-data/log-diffs/<id>.summary.txt` -- High-level summary: total changed lines, first divergence
|
|
132
|
+
point, and categorized change counts with direction (e.g. "animation frames: +85 in head /
|
|
133
|
+
-46 in base, net +39 in head").
|
|
134
|
+
- `debug-data/params-diffs/<id>.diff` -- JSON-aware diff of `launchBrowserAndReplayParams.json`
|
|
135
|
+
between head and base. Keys are sorted and pretty-printed so only meaningful value changes appear.
|
|
136
|
+
<!-- if-snapshot-assets -->
|
|
137
|
+
- `debug-data/assets-diffs/<id>.txt` -- Comparison of snapshotted asset file lists between head
|
|
138
|
+
and base (added/removed/changed by content hash). Not generated if assets are identical.
|
|
139
|
+
<!-- end-if-snapshot-assets -->
|
|
140
|
+
- `debug-data/screenshot-context/<id>.txt` -- Only generated with `--screenshot`. Shows ±30 lines
|
|
141
|
+
of `logs.deterministic.txt` surrounding the screenshot for both head and base, with the
|
|
142
|
+
screenshot line marked `>>>`.
|
|
143
|
+
- `debug-data/dom-diffs/<headReplayId>-vs-<baseReplayId>-<screenshotBaseName>.diff` -- **Start
|
|
144
|
+
here for DOM changes.** Unified diff with 3 lines of context per hunk of the HEAD vs BASE
|
|
145
|
+
DOM at a specific screenshot, identical to the diff shown in the Meticulous product. Only
|
|
146
|
+
written when the DOMs actually differ.
|
|
147
|
+
- `debug-data/dom-diffs/<headReplayId>-vs-<baseReplayId>-<screenshotBaseName>.full.diff` --
|
|
148
|
+
The same hunks as the sibling `.diff`, but with full-file context. Read this when 3 lines
|
|
149
|
+
of context around a hunk isn't enough to understand the surrounding DOM structure.
|
|
150
|
+
- `debug-data/dom-diffs/<headReplayId>-vs-<baseReplayId>.summary.txt` -- Per-pair index of DOM
|
|
151
|
+
diffs: total screenshots compared, count breakdown by status, and a TSV table (`screenshot`,
|
|
152
|
+
`status`, `hunks`, `diff_bytes`, `url`).
|
|
153
|
+
|
|
154
|
+
<!-- if-local-cli -->
|
|
155
|
+
|
|
156
|
+
Note: DOM diffs are only generated when a `replayDiffId` is available (the normal
|
|
157
|
+
`meticulous debug replay-diff <id>` path). On the rare `meticulous debug replay --baseReplayId`
|
|
158
|
+
path there is no `replayDiffId`, so `dom-diffs/` is not generated — diff the per-replay
|
|
159
|
+
`screenshots/<baseName>.html` files directly with the system `diff` command instead.
|
|
160
|
+
|
|
161
|
+
<!-- end-if-local-cli -->
|
|
162
|
+
|
|
163
|
+
Individual screenshots may also be marked `skipped-error` (backend fetch failed) or
|
|
164
|
+
`skipped-unsupported` (e.g. redacted variants, or screenshots whose identifier couldn't
|
|
165
|
+
be resolved from `timeline.json`) in the per-pair `.summary.txt`. For those, fall back
|
|
166
|
+
to diffing the two `screenshots/<baseName>.html` files directly.
|
|
167
|
+
|
|
168
|
+
### Other Data
|
|
169
|
+
|
|
170
|
+
- `debug-data/session-summaries/<sessionId>.txt` -- **Start here for session investigation.** Compact
|
|
171
|
+
summary of each session: URL history, user event breakdown, network request stats (methods,
|
|
172
|
+
status codes, domains, failures), storage counts, WebSocket connections, custom data, session
|
|
173
|
+
context, and framework info.
|
|
174
|
+
- `debug-data/sessions/<sessionId>/data.json` -- Full session recording data including user events, network
|
|
175
|
+
requests (HAR format), and application storage. Can be very large; prefer the session summary
|
|
176
|
+
or use search to find relevant portions.
|
|
177
|
+
- `debug-data/test-run/<testRunId>.json` -- Test run configuration, results, commit SHA, and status.
|
|
178
|
+
<!-- if-pr-description -->
|
|
179
|
+
- `debug-data/pr-description.txt` -- Pull request description/body.
|
|
180
|
+
<!-- end-if-pr-description -->
|
|
181
|
+
<!-- if-pr-diff -->
|
|
182
|
+
- `debug-data/pr-diff.txt` -- Source code changes between the base and head commits.
|
|
183
|
+
<!-- end-if-pr-diff -->
|
|
184
|
+
|
|
185
|
+
## Key `context.json` fields
|
|
186
|
+
|
|
187
|
+
- `screenshotMap` -- maps each screenshot identifier to its virtual timestamp and event number.
|
|
188
|
+
Use this to correlate e.g. `screenshot-after-event-00673` with a timeline position.
|
|
189
|
+
- `replayComparison` -- side-by-side per-replay stats (events, network requests, animation
|
|
190
|
+
frames, virtual time, screenshots). Scan for head-vs-base drift.
|
|
191
|
+
- `domDiffMap` -- keyed by `"<headReplayId>-vs-<baseReplayId>/<screenshotBaseName>"`. An
|
|
192
|
+
entry is present **only** for screenshots with a real DOM diff to open; each carries
|
|
193
|
+
`diffPath` (3-line-context), `fullDiffPath` (full-file-context), `totalHunks`, `bytes`,
|
|
194
|
+
and `url`. `fullDiffPath` alone can be `null` when the full-context fetch failed while
|
|
195
|
+
the canonical succeeded — in that case `diffPath` is non-null and still usable.
|
|
196
|
+
Screenshots that are identical, only-in-one-side, `skipped-error`, or
|
|
197
|
+
`skipped-unsupported` have **no entry** in the map; consult the per-pair `.summary.txt`
|
|
198
|
+
for the full list (it records every compared screenshot and its status).
|
|
199
|
+
- `fileMetadata` -- byte sizes and line counts for key files. Check this before reading
|
|
200
|
+
anything large; for files over ~5000 lines prefer grep/search or ranged reads.
|
|
201
|
+
|
|
202
|
+
## Debugging Workflow
|
|
203
|
+
|
|
204
|
+
Don't work through this top-to-bottom -- pick the phases relevant to the question. Phase 1
|
|
205
|
+
applies to every investigation; then pick phase 2 (comparing replays) or phase 3 (single
|
|
206
|
+
replay), and drop into phase 4 only as needed.
|
|
207
|
+
|
|
208
|
+
### 1. Orient
|
|
209
|
+
|
|
210
|
+
1. **Read `debug-data/context.json`** for IDs, statuses, file paths, `screenshotMap`,
|
|
211
|
+
`replayComparison`, and `fileMetadata`. If a `screenshot` field is present, that's the
|
|
212
|
+
specific screenshot the user wants to investigate -- use `screenshotMap` to find its
|
|
213
|
+
virtual timestamp and focus analysis on events leading up to it.
|
|
214
|
+
2. **Scan `replayComparison`** for head-vs-base drift signals (event counts, animation
|
|
215
|
+
frames, virtual time, screenshot count).
|
|
216
|
+
|
|
217
|
+
### 2. Investigate a diff (head vs base)
|
|
218
|
+
|
|
219
|
+
3. **Screenshot diffs** -- read `debug-data/diffs/<id>.summary.json` for which screenshots
|
|
220
|
+
changed and by how much. If `debug-data/screenshot-context/` is present, read it for the
|
|
221
|
+
log lines around each screenshot. Only open the full `diffs/<id>.json` for complete
|
|
222
|
+
replay metadata.
|
|
223
|
+
4. **DOM diffs** -- for each changed screenshot, open the matching `.diff` under
|
|
224
|
+
`debug-data/dom-diffs/` (use `domDiffMap` to navigate). If 3 lines of context around a
|
|
225
|
+
hunk isn't enough, open the sibling `.full.diff` file (same hunks with full-file context,
|
|
226
|
+
path on `domDiffMap[...].fullDiffPath`).
|
|
227
|
+
5. **Log diffs** -- **delegate to the log-diff-analyzer subagent** instead of reading
|
|
228
|
+
`debug-data/log-diffs/*.filtered.diff` directly; only open the raw diff to verify
|
|
229
|
+
specific findings.
|
|
230
|
+
<!-- if-pr-description-and-diff -->
|
|
231
|
+
6. **PR description and diff** -- read `debug-data/pr-description.txt` first to understand
|
|
232
|
+
the intended change, then **delegate to the pr-analyzer subagent** to correlate code
|
|
233
|
+
changes with visual diffs. Only open `debug-data/pr-diff.txt` directly to verify findings.
|
|
234
|
+
<!-- end-if-pr-description-and-diff -->
|
|
235
|
+
<!-- if-pr-description-only -->
|
|
236
|
+
7. **PR description** -- read `debug-data/pr-description.txt` to understand the intended
|
|
237
|
+
change that triggered this test run.
|
|
238
|
+
<!-- end-if-pr-description-only -->
|
|
239
|
+
<!-- if-pr-diff-only -->
|
|
240
|
+
8. **PR diff** -- **delegate to the pr-analyzer subagent** to correlate code changes with
|
|
241
|
+
visual diffs. Only open `debug-data/pr-diff.txt` directly to verify findings.
|
|
242
|
+
<!-- end-if-pr-diff-only -->
|
|
243
|
+
|
|
244
|
+
### 3. Investigate a single replay
|
|
245
|
+
|
|
246
|
+
7. **Filtered logs** -- read `logs.deterministic.filtered.txt` inside the replay directory.
|
|
247
|
+
Fall back to the raw `logs.deterministic.txt` only if you need unmodified output.
|
|
248
|
+
8. **Events index** -- grep `debug-data/events-index/<role>-<replayId>.txt` by kind,
|
|
249
|
+
virtual time, or URL (e.g. `grep 'kind=screenshot'`, `grep 'api/v9/users'`). Prefer this
|
|
250
|
+
over parsing `timeline.json`.
|
|
251
|
+
9. **Logs index** -- grep `debug-data/logs-index/<role>-<replayId>.txt` by source
|
|
252
|
+
(`source=application`), type (`type=warn`), keyword, or virtual time (`vt=1234`).
|
|
253
|
+
|
|
254
|
+
### 3b. Check network match quality (when diffs involve partial matches)
|
|
255
|
+
|
|
256
|
+
When the replay shows "Low accuracy: Network issues", or `network-log/` shows pollyReplay
|
|
257
|
+
entries with a `repair=` suffix (indicating session repair was needed to serve a response)
|
|
258
|
+
in the root cause time span (events preceding the first visual difference), this is a
|
|
259
|
+
high-priority check:
|
|
260
|
+
|
|
261
|
+
1. Grep `timeline.ndjson` for `pollyReplay` events near the first divergence point. Each
|
|
262
|
+
event's `data` contains both:
|
|
263
|
+
- `data.pollyRequest.request.url` -- the **actual request** made during replay.
|
|
264
|
+
- `data.matchedRequest.request.url` -- the **original recorded request** whose saved
|
|
265
|
+
response was served (null if no match was found).
|
|
266
|
+
2. Compare these URLs for each pollyReplay event. If they differ significantly (different
|
|
267
|
+
endpoint, different query parameters, different resource), the response served almost
|
|
268
|
+
certainly doesn't match what the application expected. This is a common root cause for
|
|
269
|
+
diffs and should be flagged.
|
|
270
|
+
|
|
271
|
+
For example, if the actual request was `GET /api/v1/items` but Meticulous served a saved
|
|
272
|
+
response originally recorded for `GET /api/v1/orders?page=1&search=...`, the
|
|
273
|
+
response schema is likely wrong for the requesting code, causing downstream UI differences.
|
|
274
|
+
|
|
275
|
+
When you identify such a mismatch, report it as the likely root cause and note that the diff
|
|
276
|
+
can be disregarded -- it stems from a network matching issue, not a real code regression.
|
|
277
|
+
|
|
278
|
+
### 4. Deeper dives (as needed)
|
|
279
|
+
|
|
280
|
+
10. **Screenshot timeline context** -- `debug-data/screenshot-timeline-context/` for the 30
|
|
281
|
+
events before and 10 after each screenshot.
|
|
282
|
+
11. **Network activity** -- grep `debug-data/network-log/` for endpoints, status codes,
|
|
283
|
+
domains. For any pollyReplay entries with a `repair=` suffix, cross-reference with
|
|
284
|
+
`timeline.ndjson` to compare the actual request URL against the matched recorded request
|
|
285
|
+
URL (see phase 3b above).
|
|
286
|
+
12. **Virtual time progression** -- `diff` on `debug-data/vt-progression/` files to find
|
|
287
|
+
where head and base diverge.
|
|
288
|
+
13. **Replay parameters** -- `debug-data/params-diffs/` for computed diffs, or
|
|
289
|
+
`launchBrowserAndReplayParams.json` for a single replay.
|
|
290
|
+
14. **Code coverage** -- `mapped-coverage.json` for whole-replay coverage; correlate executed
|
|
291
|
+
code with `pr-diff.txt` to see whether the changed lines actually ran. Use
|
|
292
|
+
`mapped-per-screenshot-js-coverage/<screenshotId>.json` to localize what executed before
|
|
293
|
+
a specific screenshot.
|
|
294
|
+
<!-- if-snapshot-assets -->
|
|
295
|
+
15. **Assets** -- `debug-data/assets-diffs/` for snapshotted JS/CSS diffs,
|
|
296
|
+
`debug-data/formatted-assets/` for pretty-printed bundles.
|
|
297
|
+
<!-- end-if-snapshot-assets -->
|
|
298
|
+
16. **Session data** -- `debug-data/session-summaries/<sessionId>.txt` first; only read raw
|
|
299
|
+
`sessions/<id>/data.json` for specific request bodies or event selectors.
|
|
300
|
+
<!-- if-local-cli -->
|
|
301
|
+
17. **Project source** -- `project-repo/` when present.
|
|
302
|
+
<!-- if-snapshot-assets -->
|
|
303
|
+
For third-party library code, use `debug-data/formatted-assets/`.
|
|
304
|
+
<!-- end-if-snapshot-assets -->
|
|
305
|
+
<!-- end-if-local-cli -->
|
|
306
|
+
|
|
307
|
+
**Important**: Do NOT use Python one-liners to parse `timeline.json` or `logs.ndjson`. The
|
|
308
|
+
derived files above (`events-index/`, `logs-index/`, `network-log/`, `vt-progression/`,
|
|
309
|
+
`screenshot-timeline-context/`) are pre-computed and greppable. Use `timeline.ndjson`
|
|
310
|
+
(NDJSON format, one JSON object per line) if you need to grep the raw timeline data.
|
|
311
|
+
|
|
312
|
+
## Subagents
|
|
313
|
+
|
|
314
|
+
Specialized subagents are available via the Task tool — the SDK surfaces each one's
|
|
315
|
+
name, tools, and description automatically; you don't need to enumerate them. Two
|
|
316
|
+
general-purpose subagents are worth calling out because they apply across all phases
|
|
317
|
+
of the workflow:
|
|
318
|
+
|
|
319
|
+
- **Planner** — when the user describes a complex or ambiguous issue, delegate to
|
|
320
|
+
the planner before starting your own investigation. It reads workspace summaries
|
|
321
|
+
and metadata to produce a structured debugging plan with prioritized investigation
|
|
322
|
+
steps. Skip it for straightforward cases (e.g. a specific screenshot diff with an
|
|
323
|
+
obvious focus, or a single diff to investigate).
|
|
324
|
+
- **Summarizer** — when you need to understand a large file (over 5000 lines),
|
|
325
|
+
delegate to the summarizer instead of reading the file in full. It scans with
|
|
326
|
+
grep and targeted reads and returns a concise overview with line numbers for
|
|
327
|
+
follow-up.
|
|
328
|
+
|
|
329
|
+
Phase-specific subagents (log-diff-analyzer, pr-analyzer) are referenced from the
|
|
330
|
+
workflow steps above.
|
|
331
|
+
|
|
332
|
+
## Rules
|
|
333
|
+
|
|
334
|
+
- This workspace is for analysis and investigation. Focus on understanding root causes.
|
|
335
|
+
- When referencing files, use paths relative to this workspace root.
|
|
336
|
+
- Prefer `logs.deterministic.filtered.txt` over `logs.deterministic.txt` for general
|
|
337
|
+
investigation. Use the raw version only when you need unmodified output.
|
|
338
|
+
- Prefer `logs.deterministic.txt` over `logs.concise.txt` when comparing between replays,
|
|
339
|
+
since real-time timestamps are stripped.
|
|
340
|
+
- Session data files can be very large. Use grep/search to find relevant portions rather than
|
|
341
|
+
reading entire files.
|
|
342
|
+
- For per-screenshot DOM content, prefer reading
|
|
343
|
+
`debug-data/replays/<role>/<replayId>/screenshots/<name>.html` rather than parsing the raw
|
|
344
|
+
`<name>.metadata.json`.
|
|
345
|
+
- For DOM changes between replays, prefer `debug-data/dom-diffs/` over diffing the two
|
|
346
|
+
`<name>.html` files yourself -- the pre-computed `.diff` is identical to the diff shown
|
|
347
|
+
in the Meticulous product and already has context lines added.
|
|
348
|
+
- Check `fileMetadata` in `context.json` for file sizes before reading large files.
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: log-diff-analyzer
|
|
3
|
+
description: Analyzes filtered log diffs between head and base replays to identify divergence patterns and form hypotheses. Use when comparing replays and log diffs are available.
|
|
4
|
+
tools: Read, Grep, Glob
|
|
5
|
+
model: sonnet
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
You are a log diff analysis specialist for debugging Meticulous replay issues.
|
|
9
|
+
|
|
10
|
+
When delegated to, analyze the filtered log diffs to identify where and why the head and
|
|
11
|
+
base replays diverged. Produce a structured report the main agent can act on without
|
|
12
|
+
reading the raw diffs.
|
|
13
|
+
|
|
14
|
+
## Process
|
|
15
|
+
|
|
16
|
+
1. Read `debug-data/log-diffs/*.summary.txt` for the pre-computed overview: total changed
|
|
17
|
+
lines, first divergence point, and categorized change counts.
|
|
18
|
+
2. Read `debug-data/log-diffs/*.filtered.diff`. If the diff exceeds ~3000 lines, use Grep
|
|
19
|
+
to focus on key patterns:
|
|
20
|
+
- First divergence: read ~100 lines around the first hunk
|
|
21
|
+
- Network changes: grep for `fetch`, `request`, `response`, `network`
|
|
22
|
+
- Animation frames: grep for `animation`, `requestAnimationFrame`, `rAF`, `jsReplay`
|
|
23
|
+
- Timers: grep for `setTimeout`, `setInterval`, `timer`, `tick`
|
|
24
|
+
- Screenshots: grep for `screenshot`
|
|
25
|
+
- Errors: grep for `error`, `warning`, `fail`, `timeout`
|
|
26
|
+
- Navigation: grep for `navigation`, `navigate`, `pushState`
|
|
27
|
+
3. Read `debug-data/params-diffs/*.diff` if present, to check for configuration differences.
|
|
28
|
+
|
|
29
|
+
## Output Format
|
|
30
|
+
|
|
31
|
+
Return a structured analysis under 800 words:
|
|
32
|
+
|
|
33
|
+
### Divergence Overview
|
|
34
|
+
|
|
35
|
+
- Total changed lines (added/removed) and first divergence line number
|
|
36
|
+
- Whether changes are concentrated in one area or spread throughout
|
|
37
|
+
- Whether changes are dominated by one category or mixed
|
|
38
|
+
|
|
39
|
+
### Change Breakdown
|
|
40
|
+
|
|
41
|
+
For each category with significant changes:
|
|
42
|
+
|
|
43
|
+
- **Category name** (network, animation, timers, navigation, screenshots, errors, other)
|
|
44
|
+
- Line count and direction (+N in head / -N in base)
|
|
45
|
+
- Representative examples with line numbers from the diff
|
|
46
|
+
- Whether the changes in this category could explain the screenshot diffs
|
|
47
|
+
|
|
48
|
+
### First Divergence Point
|
|
49
|
+
|
|
50
|
+
- Exact line number and context (5-10 lines before and after)
|
|
51
|
+
- What event or action triggered the divergence
|
|
52
|
+
- Whether this is a root cause or a downstream effect
|
|
53
|
+
|
|
54
|
+
### Hypotheses (ranked by likelihood)
|
|
55
|
+
|
|
56
|
+
For each hypothesis:
|
|
57
|
+
|
|
58
|
+
- One-line description (e.g., "Animation frame count drift from continuous Lottie animation")
|
|
59
|
+
- Supporting evidence from the diff
|
|
60
|
+
- What the main agent should check to confirm or rule it out
|
|
61
|
+
- Specific line ranges or grep patterns for follow-up
|
|
62
|
+
|
|
63
|
+
### Recommended Next Steps
|
|
64
|
+
|
|
65
|
+
Ordered list of what the main agent should investigate, with specific file paths and
|
|
66
|
+
line numbers.
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: planner
|
|
3
|
+
description: Creates a structured debugging plan based on workspace data and user context. Use proactively at the start of every debugging session after the user describes their issue.
|
|
4
|
+
tools: Read, Grep, Glob
|
|
5
|
+
model: sonnet
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
You are a debugging planning assistant for the Meticulous automated UI testing platform.
|
|
9
|
+
|
|
10
|
+
Your job is to quickly scan the workspace data and produce a structured debugging plan
|
|
11
|
+
that the main agent will follow. You run at the start of a session after the developer
|
|
12
|
+
describes the issue they want to investigate.
|
|
13
|
+
|
|
14
|
+
## What to Read
|
|
15
|
+
|
|
16
|
+
Gather context from these sources (in order):
|
|
17
|
+
|
|
18
|
+
1. `context.json` -- IDs, file paths, `screenshotMap`, `replayComparison`, `fileMetadata`.
|
|
19
|
+
If a `screenshot` field is present, the developer wants to investigate that specific
|
|
20
|
+
screenshot.
|
|
21
|
+
2. `timeline-summaries/*.txt` -- compact overview of each replay's events, screenshot
|
|
22
|
+
timestamps, and counts.
|
|
23
|
+
3. `log-diffs/*.summary.txt` -- high-level log diff summary with categorized change counts
|
|
24
|
+
(only present when comparing replays).
|
|
25
|
+
4. `diffs/*.summary.json` -- which screenshots differ and by how much (only present when
|
|
26
|
+
comparing replays).
|
|
27
|
+
5. `params-diffs/*.diff` -- parameter differences between head and base replays.
|
|
28
|
+
<!-- if-pr-diff -->
|
|
29
|
+
6. `pr-diff.txt` -- source code changes (first ~200 lines if large).
|
|
30
|
+
<!-- end-if-pr-diff -->
|
|
31
|
+
|
|
32
|
+
## What to Produce
|
|
33
|
+
|
|
34
|
+
Based on the workspace data and the developer's description, output:
|
|
35
|
+
|
|
36
|
+
### Initial Assessment
|
|
37
|
+
|
|
38
|
+
- What type of issue is this? (flake, unexpected diff, replay failure, investigation)
|
|
39
|
+
- What data is available in the workspace?
|
|
40
|
+
- Key observations from summaries and comparisons (e.g. event count drift, virtual time
|
|
41
|
+
differences, screenshot mismatch percentages).
|
|
42
|
+
|
|
43
|
+
### Investigation Steps (ordered by priority)
|
|
44
|
+
|
|
45
|
+
For each step:
|
|
46
|
+
|
|
47
|
+
- What to examine and why
|
|
48
|
+
- Specific file paths to read
|
|
49
|
+
- What patterns or anomalies to look for
|
|
50
|
+
- What would confirm or rule out each hypothesis
|
|
51
|
+
|
|
52
|
+
### Key Files
|
|
53
|
+
|
|
54
|
+
List the most important files with their sizes (from `fileMetadata` in `context.json`).
|
|
55
|
+
Flag any files too large to read in full and suggest using the summarizer subagent or
|
|
56
|
+
grep for those.
|
|
57
|
+
|
|
58
|
+
## Guidelines
|
|
59
|
+
|
|
60
|
+
- Be concise. The plan should be actionable, not exhaustive.
|
|
61
|
+
- Prioritize the most likely root causes first.
|
|
62
|
+
- If the developer mentioned a specific screenshot, correlate it with the `screenshotMap`
|
|
63
|
+
to find its virtual timestamp and event number, and focus the plan around events leading
|
|
64
|
+
up to that screenshot.
|
|
65
|
+
- If `replayComparison` shows drift (different event counts, animation frames, or virtual
|
|
66
|
+
time), call that out prominently.
|
|
67
|
+
- Suggest which debugging skills (in `.claude/skills/`) are most relevant to the issue.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: pr-analyzer
|
|
3
|
+
description: Analyzes PR source code changes and correlates them with screenshot diffs. Use when pr-diff.txt is present and you need to understand which code changes caused visual differences.
|
|
4
|
+
tools: Read, Grep, Glob
|
|
5
|
+
model: sonnet
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
You are a PR analysis specialist for debugging Meticulous replay issues.
|
|
9
|
+
|
|
10
|
+
When delegated to, analyze the source code changes in the PR and correlate them with the
|
|
11
|
+
screenshot diffs to determine which code changes likely caused which visual differences.
|
|
12
|
+
|
|
13
|
+
## Process
|
|
14
|
+
|
|
15
|
+
1. Read `debug-data/diffs/*.summary.json` to understand which screenshots differ, their
|
|
16
|
+
mismatch percentages, and their `changedSectionsClassNames`.
|
|
17
|
+
2. Read `debug-data/pr-diff.txt`. If it exceeds ~2000 lines, use Grep to find the most
|
|
18
|
+
relevant sections:
|
|
19
|
+
- CSS/SCSS/Tailwind changes: `className`, `style`, `css`, `scss`, `tailwind`
|
|
20
|
+
- Component rendering: `return`, `render`, `jsx`, `tsx`
|
|
21
|
+
- Layout changes: `flex`, `grid`, `position`, `margin`, `padding`, `width`, `height`
|
|
22
|
+
- Visibility: `display`, `hidden`, `visible`, `opacity`
|
|
23
|
+
3. Cross-reference the `changedSectionsClassNames` from the diff summaries with class names
|
|
24
|
+
and component names in the PR diff to establish causal links.
|
|
25
|
+
|
|
26
|
+
## Output Format
|
|
27
|
+
|
|
28
|
+
Return a structured analysis under 800 words:
|
|
29
|
+
|
|
30
|
+
### Files Changed
|
|
31
|
+
|
|
32
|
+
List all changed files with +/- line counts, grouped by category:
|
|
33
|
+
|
|
34
|
+
- **Visual/Styling**: CSS, SCSS, component files with UI changes
|
|
35
|
+
- **Logic/Data**: API calls, state management, utilities
|
|
36
|
+
- **Config/Other**: Package files, configs, tests
|
|
37
|
+
|
|
38
|
+
### Code-to-Diff Correlation
|
|
39
|
+
|
|
40
|
+
For each screenshot that differs (from the diff summaries):
|
|
41
|
+
|
|
42
|
+
- Screenshot identifier and mismatch percentage
|
|
43
|
+
- The `changedSectionsClassNames` from the diff
|
|
44
|
+
- Which PR file(s) most likely caused this diff, with specific line references
|
|
45
|
+
- Whether this visual change appears **expected** (intentional, matches code intent) or
|
|
46
|
+
**unexpected** (no clear code change correspondence)
|
|
47
|
+
|
|
48
|
+
### Uncorrelated Changes
|
|
49
|
+
|
|
50
|
+
Note any PR changes that should affect visual output but don't appear in any screenshot
|
|
51
|
+
diffs (possible coverage gaps), and any screenshot diffs that don't correspond to any
|
|
52
|
+
obvious code change (possible flakes or indirect effects).
|
|
53
|
+
|
|
54
|
+
### Summary
|
|
55
|
+
|
|
56
|
+
One paragraph: what is this PR doing, which visual changes are expected, and which (if any)
|
|
57
|
+
warrant further investigation?
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: summarizer
|
|
3
|
+
description: Summarizes large files (logs, timelines, session data, diffs) that are too large to read in full. Use when a file exceeds 5000 lines or when you need a quick overview of a large file's contents.
|
|
4
|
+
tools: Read, Grep, Glob
|
|
5
|
+
model: haiku
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
You are a file summarization specialist for debugging Meticulous replay issues.
|
|
9
|
+
|
|
10
|
+
When given a file to summarize, produce a concise overview that helps the main agent
|
|
11
|
+
decide what to investigate further. Do not read the entire file -- use Grep and targeted
|
|
12
|
+
reads to extract the key information efficiently.
|
|
13
|
+
|
|
14
|
+
## Process
|
|
15
|
+
|
|
16
|
+
1. Read the first ~50 lines to understand the file's structure and format.
|
|
17
|
+
2. Use Grep to find key patterns: errors, warnings, screenshots, network failures,
|
|
18
|
+
timeouts, navigation events, and any terms the caller highlighted.
|
|
19
|
+
3. Read targeted sections around important matches.
|
|
20
|
+
4. Read the last ~30 lines for final state or summary information.
|
|
21
|
+
5. Produce a structured summary.
|
|
22
|
+
|
|
23
|
+
## File-Type Guidelines
|
|
24
|
+
|
|
25
|
+
### Log files (`logs.deterministic.txt`, `logs.deterministic.filtered.txt`, `logs.concise.txt`)
|
|
26
|
+
|
|
27
|
+
Summarize:
|
|
28
|
+
|
|
29
|
+
- Approximate line count and virtual time range
|
|
30
|
+
- Key phases: navigation, network loading, user events, screenshots
|
|
31
|
+
- Errors, warnings, or unusual patterns (grep for `error`, `warning`, `fail`, `timeout`)
|
|
32
|
+
- Network request overview: grep for `request` and note counts, failures
|
|
33
|
+
- Screenshot timestamps and event numbers
|
|
34
|
+
|
|
35
|
+
### Timeline files (`timeline.json`)
|
|
36
|
+
|
|
37
|
+
Summarize:
|
|
38
|
+
|
|
39
|
+
- Total entry count
|
|
40
|
+
- Event kind breakdown (grep for `"kind":` and tally)
|
|
41
|
+
- Any `potentialFlakinessWarning` entries
|
|
42
|
+
- Virtual time range (first and last entries)
|
|
43
|
+
- Notable gaps or clusters of events
|
|
44
|
+
|
|
45
|
+
### Session data (`sessions/*/data.json`)
|
|
46
|
+
|
|
47
|
+
Summarize:
|
|
48
|
+
|
|
49
|
+
- Session structure (grep for top-level keys)
|
|
50
|
+
- User interaction count and types
|
|
51
|
+
- Network request patterns (count, domains)
|
|
52
|
+
- Any storage or cookie data of note
|
|
53
|
+
|
|
54
|
+
### Diff files (`log-diffs/*.diff`, `log-diffs/*.filtered.diff`)
|
|
55
|
+
|
|
56
|
+
Summarize:
|
|
57
|
+
|
|
58
|
+
- Total hunks and changed line counts
|
|
59
|
+
- Categories of changes (network, animation, timers, navigation)
|
|
60
|
+
- Location of first divergence
|
|
61
|
+
- Whether changes are concentrated or spread throughout
|
|
62
|
+
|
|
63
|
+
### Any other file
|
|
64
|
+
|
|
65
|
+
Summarize:
|
|
66
|
+
|
|
67
|
+
- File structure and format
|
|
68
|
+
- Size and key sections
|
|
69
|
+
- Notable content relevant to debugging
|
|
70
|
+
|
|
71
|
+
## Output Format
|
|
72
|
+
|
|
73
|
+
Return a summary under 500 words. Include specific line numbers or grep patterns so
|
|
74
|
+
the main agent can follow up on anything interesting. Structure the summary with clear
|
|
75
|
+
headings for easy scanning.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
{
|
|
2
|
+
"permissions": {
|
|
3
|
+
"allow": [
|
|
4
|
+
"Read",
|
|
5
|
+
"Grep",
|
|
6
|
+
"Glob",
|
|
7
|
+
"Task",
|
|
8
|
+
"Bash(find *)",
|
|
9
|
+
"Bash(grep *)",
|
|
10
|
+
"Bash(ls *)",
|
|
11
|
+
"Bash(git diff *)",
|
|
12
|
+
"Bash(git log *)",
|
|
13
|
+
"Bash(git show *)",
|
|
14
|
+
"Bash(git status)",
|
|
15
|
+
"Bash(git rev-parse *)",
|
|
16
|
+
"Bash(git blame *)",
|
|
17
|
+
"Bash(git grep *)",
|
|
18
|
+
"Bash(diff *)",
|
|
19
|
+
"Bash(wc *)",
|
|
20
|
+
"Bash(head *)",
|
|
21
|
+
"Bash(tail *)",
|
|
22
|
+
"Bash(sort *)",
|
|
23
|
+
"Bash(uniq *)",
|
|
24
|
+
"Bash(jq *)",
|
|
25
|
+
"Bash(cat *)"
|
|
26
|
+
],
|
|
27
|
+
"ask": ["WebFetch", "WebSearch"],
|
|
28
|
+
"deny": [
|
|
29
|
+
"Edit",
|
|
30
|
+
"Write",
|
|
31
|
+
"mcp__*",
|
|
32
|
+
"ToolSearch",
|
|
33
|
+
"ListMcpResourcesTool",
|
|
34
|
+
"ReadMcpResourceTool"
|
|
35
|
+
],
|
|
36
|
+
"defaultMode": "default"
|
|
37
|
+
},
|
|
38
|
+
"spinnerVerbs": {
|
|
39
|
+
"mode": "replace",
|
|
40
|
+
"verbs": [
|
|
41
|
+
"Analyzing",
|
|
42
|
+
"Investigating",
|
|
43
|
+
"Comparing",
|
|
44
|
+
"Searching",
|
|
45
|
+
"Processing",
|
|
46
|
+
"Squinting at screenshots",
|
|
47
|
+
"Traversing tangled timelines",
|
|
48
|
+
"Fighting fickle flakes",
|
|
49
|
+
"Diagnosing dubious diffs",
|
|
50
|
+
"Reconciling replays",
|
|
51
|
+
"Deciphering devious divergences",
|
|
52
|
+
"Debugging flakes",
|
|
53
|
+
"Patching network sessions",
|
|
54
|
+
"Determining nondeterminism"
|
|
55
|
+
]
|
|
56
|
+
}
|
|
57
|
+
}
|