@alwaysmeticulous/debug-workspace 2.306.0 → 2.307.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/__tests__/detect-snapshot-assets.spec.js +4 -1
- package/dist/__tests__/detect-snapshot-assets.spec.js.map +1 -1
- package/dist/__tests__/extract-screenshot-dom-files.spec.js +4 -1
- package/dist/__tests__/extract-screenshot-dom-files.spec.js.map +1 -1
- package/dist/__tests__/fetch-dom-diffs.spec.js +5 -1
- package/dist/__tests__/fetch-dom-diffs.spec.js.map +1 -1
- package/dist/__tests__/meticulous-sha.spec.d.ts +2 -0
- package/dist/__tests__/meticulous-sha.spec.d.ts.map +1 -0
- package/dist/__tests__/meticulous-sha.spec.js +47 -0
- package/dist/__tests__/meticulous-sha.spec.js.map +1 -0
- package/dist/__tests__/redact-cookies-and-headers.spec.js +5 -1
- package/dist/__tests__/redact-cookies-and-headers.spec.js.map +1 -1
- package/dist/__tests__/redact-log-credentials.spec.js +4 -1
- package/dist/__tests__/redact-log-credentials.spec.js.map +1 -1
- package/dist/debug-constants.js +4 -1
- package/dist/debug-constants.js.map +1 -1
- package/dist/debug.types.d.ts +1 -0
- package/dist/debug.types.d.ts.map +1 -1
- package/dist/debug.types.js +4 -1
- package/dist/debug.types.js.map +1 -1
- package/dist/download-debug-data.js +4 -1
- package/dist/download-debug-data.js.map +1 -1
- package/dist/extract-screenshot-dom-files.js +4 -1
- package/dist/extract-screenshot-dom-files.js.map +1 -1
- package/dist/fetch-dom-diffs.js +4 -1
- package/dist/fetch-dom-diffs.js.map +1 -1
- package/dist/generate-debug-derived-files.js +6 -2
- package/dist/generate-debug-derived-files.js.map +1 -1
- package/dist/generate-debug-workspace.d.ts.map +1 -1
- package/dist/generate-debug-workspace.js +11 -1
- package/dist/generate-debug-workspace.js.map +1 -1
- package/dist/index.d.ts +1 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +8 -2
- package/dist/index.js.map +1 -1
- package/dist/meticulous-sha.d.ts +7 -0
- package/dist/meticulous-sha.d.ts.map +1 -0
- package/dist/meticulous-sha.js +26 -0
- package/dist/meticulous-sha.js.map +1 -0
- package/dist/pipeline.js +4 -1
- package/dist/pipeline.js.map +1 -1
- package/dist/replay-walk.js +4 -1
- package/dist/replay-walk.js.map +1 -1
- package/dist/resolve-debug-context.js +6 -1
- package/dist/resolve-debug-context.js.map +1 -1
- package/dist/screenshot-identifier.d.ts +6 -2
- package/dist/screenshot-identifier.d.ts.map +1 -1
- package/dist/screenshot-identifier.js +23 -3
- package/dist/screenshot-identifier.js.map +1 -1
- package/dist/screenshot-identifier.spec.d.ts +2 -0
- package/dist/screenshot-identifier.spec.d.ts.map +1 -0
- package/dist/screenshot-identifier.spec.js +53 -0
- package/dist/screenshot-identifier.spec.js.map +1 -0
- package/dist/templates/CLAUDE.md +6 -1
- package/package.json +4 -4
- package/dist/templates/templates/CLAUDE.md +0 -348
- package/dist/templates/templates/agents/log-diff-analyzer.md +0 -66
- package/dist/templates/templates/agents/planner.md +0 -67
- package/dist/templates/templates/agents/pr-analyzer.md +0 -57
- package/dist/templates/templates/agents/summarizer.md +0 -75
- package/dist/templates/templates/settings.json +0 -57
- package/dist/templates/templates/skills/debugging-diffs/SKILL.md +0 -61
- package/dist/templates/templates/skills/debugging-flakes/SKILL.md +0 -52
- package/dist/templates/templates/skills/debugging-network/SKILL.md +0 -45
- package/dist/templates/templates/skills/debugging-sessions/SKILL.md +0 -47
- package/dist/templates/templates/skills/debugging-timelines/SKILL.md +0 -51
- package/dist/templates/templates/skills/pr-analysis/SKILL.md +0 -20
|
@@ -1,66 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: log-diff-analyzer
|
|
3
|
-
description: Analyzes filtered log diffs between head and base replays to identify divergence patterns and form hypotheses. Use when comparing replays and log diffs are available.
|
|
4
|
-
tools: Read, Grep, Glob
|
|
5
|
-
model: sonnet
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
You are a log diff analysis specialist for debugging Meticulous replay issues.
|
|
9
|
-
|
|
10
|
-
When delegated to, analyze the filtered log diffs to identify where and why the head and
|
|
11
|
-
base replays diverged. Produce a structured report the main agent can act on without
|
|
12
|
-
reading the raw diffs.
|
|
13
|
-
|
|
14
|
-
## Process
|
|
15
|
-
|
|
16
|
-
1. Read `debug-data/log-diffs/*.summary.txt` for the pre-computed overview: total changed
|
|
17
|
-
lines, first divergence point, and categorized change counts.
|
|
18
|
-
2. Read `debug-data/log-diffs/*.filtered.diff`. If the diff exceeds ~3000 lines, use Grep
|
|
19
|
-
to focus on key patterns:
|
|
20
|
-
- First divergence: read ~100 lines around the first hunk
|
|
21
|
-
- Network changes: grep for `fetch`, `request`, `response`, `network`
|
|
22
|
-
- Animation frames: grep for `animation`, `requestAnimationFrame`, `rAF`, `jsReplay`
|
|
23
|
-
- Timers: grep for `setTimeout`, `setInterval`, `timer`, `tick`
|
|
24
|
-
- Screenshots: grep for `screenshot`
|
|
25
|
-
- Errors: grep for `error`, `warning`, `fail`, `timeout`
|
|
26
|
-
- Navigation: grep for `navigation`, `navigate`, `pushState`
|
|
27
|
-
3. Read `debug-data/params-diffs/*.diff` if present, to check for configuration differences.
|
|
28
|
-
|
|
29
|
-
## Output Format
|
|
30
|
-
|
|
31
|
-
Return a structured analysis under 800 words:
|
|
32
|
-
|
|
33
|
-
### Divergence Overview
|
|
34
|
-
|
|
35
|
-
- Total changed lines (added/removed) and first divergence line number
|
|
36
|
-
- Whether changes are concentrated in one area or spread throughout
|
|
37
|
-
- Whether changes are dominated by one category or mixed
|
|
38
|
-
|
|
39
|
-
### Change Breakdown
|
|
40
|
-
|
|
41
|
-
For each category with significant changes:
|
|
42
|
-
|
|
43
|
-
- **Category name** (network, animation, timers, navigation, screenshots, errors, other)
|
|
44
|
-
- Line count and direction (+N in head / -N in base)
|
|
45
|
-
- Representative examples with line numbers from the diff
|
|
46
|
-
- Whether the changes in this category could explain the screenshot diffs
|
|
47
|
-
|
|
48
|
-
### First Divergence Point
|
|
49
|
-
|
|
50
|
-
- Exact line number and context (5-10 lines before and after)
|
|
51
|
-
- What event or action triggered the divergence
|
|
52
|
-
- Whether this is a root cause or a downstream effect
|
|
53
|
-
|
|
54
|
-
### Hypotheses (ranked by likelihood)
|
|
55
|
-
|
|
56
|
-
For each hypothesis:
|
|
57
|
-
|
|
58
|
-
- One-line description (e.g., "Animation frame count drift from continuous Lottie animation")
|
|
59
|
-
- Supporting evidence from the diff
|
|
60
|
-
- What the main agent should check to confirm or rule it out
|
|
61
|
-
- Specific line ranges or grep patterns for follow-up
|
|
62
|
-
|
|
63
|
-
### Recommended Next Steps
|
|
64
|
-
|
|
65
|
-
Ordered list of what the main agent should investigate, with specific file paths and
|
|
66
|
-
line numbers.
|
|
@@ -1,67 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: planner
|
|
3
|
-
description: Creates a structured debugging plan based on workspace data and user context. Use proactively at the start of every debugging session after the user describes their issue.
|
|
4
|
-
tools: Read, Grep, Glob
|
|
5
|
-
model: sonnet
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
You are a debugging planning assistant for the Meticulous automated UI testing platform.
|
|
9
|
-
|
|
10
|
-
Your job is to quickly scan the workspace data and produce a structured debugging plan
|
|
11
|
-
that the main agent will follow. You run at the start of a session after the developer
|
|
12
|
-
describes the issue they want to investigate.
|
|
13
|
-
|
|
14
|
-
## What to Read
|
|
15
|
-
|
|
16
|
-
Gather context from these sources (in order):
|
|
17
|
-
|
|
18
|
-
1. `context.json` -- IDs, file paths, `screenshotMap`, `replayComparison`, `fileMetadata`.
|
|
19
|
-
If a `screenshot` field is present, the developer wants to investigate that specific
|
|
20
|
-
screenshot.
|
|
21
|
-
2. `timeline-summaries/*.txt` -- compact overview of each replay's events, screenshot
|
|
22
|
-
timestamps, and counts.
|
|
23
|
-
3. `log-diffs/*.summary.txt` -- high-level log diff summary with categorized change counts
|
|
24
|
-
(only present when comparing replays).
|
|
25
|
-
4. `diffs/*.summary.json` -- which screenshots differ and by how much (only present when
|
|
26
|
-
comparing replays).
|
|
27
|
-
5. `params-diffs/*.diff` -- parameter differences between head and base replays.
|
|
28
|
-
<!-- if-pr-diff -->
|
|
29
|
-
6. `pr-diff.txt` -- source code changes (first ~200 lines if large).
|
|
30
|
-
<!-- end-if-pr-diff -->
|
|
31
|
-
|
|
32
|
-
## What to Produce
|
|
33
|
-
|
|
34
|
-
Based on the workspace data and the developer's description, output:
|
|
35
|
-
|
|
36
|
-
### Initial Assessment
|
|
37
|
-
|
|
38
|
-
- What type of issue is this? (flake, unexpected diff, replay failure, investigation)
|
|
39
|
-
- What data is available in the workspace?
|
|
40
|
-
- Key observations from summaries and comparisons (e.g. event count drift, virtual time
|
|
41
|
-
differences, screenshot mismatch percentages).
|
|
42
|
-
|
|
43
|
-
### Investigation Steps (ordered by priority)
|
|
44
|
-
|
|
45
|
-
For each step:
|
|
46
|
-
|
|
47
|
-
- What to examine and why
|
|
48
|
-
- Specific file paths to read
|
|
49
|
-
- What patterns or anomalies to look for
|
|
50
|
-
- What would confirm or rule out each hypothesis
|
|
51
|
-
|
|
52
|
-
### Key Files
|
|
53
|
-
|
|
54
|
-
List the most important files with their sizes (from `fileMetadata` in `context.json`).
|
|
55
|
-
Flag any files too large to read in full and suggest using the summarizer subagent or
|
|
56
|
-
grep for those.
|
|
57
|
-
|
|
58
|
-
## Guidelines
|
|
59
|
-
|
|
60
|
-
- Be concise. The plan should be actionable, not exhaustive.
|
|
61
|
-
- Prioritize the most likely root causes first.
|
|
62
|
-
- If the developer mentioned a specific screenshot, correlate it with the `screenshotMap`
|
|
63
|
-
to find its virtual timestamp and event number, and focus the plan around events leading
|
|
64
|
-
up to that screenshot.
|
|
65
|
-
- If `replayComparison` shows drift (different event counts, animation frames, or virtual
|
|
66
|
-
time), call that out prominently.
|
|
67
|
-
- Suggest which debugging skills (in `.claude/skills/`) are most relevant to the issue.
|
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: pr-analyzer
|
|
3
|
-
description: Analyzes PR source code changes and correlates them with screenshot diffs. Use when pr-diff.txt is present and you need to understand which code changes caused visual differences.
|
|
4
|
-
tools: Read, Grep, Glob
|
|
5
|
-
model: sonnet
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
You are a PR analysis specialist for debugging Meticulous replay issues.
|
|
9
|
-
|
|
10
|
-
When delegated to, analyze the source code changes in the PR and correlate them with the
|
|
11
|
-
screenshot diffs to determine which code changes likely caused which visual differences.
|
|
12
|
-
|
|
13
|
-
## Process
|
|
14
|
-
|
|
15
|
-
1. Read `debug-data/diffs/*.summary.json` to understand which screenshots differ, their
|
|
16
|
-
mismatch percentages, and their `changedSectionsClassNames`.
|
|
17
|
-
2. Read `debug-data/pr-diff.txt`. If it exceeds ~2000 lines, use Grep to find the most
|
|
18
|
-
relevant sections:
|
|
19
|
-
- CSS/SCSS/Tailwind changes: `className`, `style`, `css`, `scss`, `tailwind`
|
|
20
|
-
- Component rendering: `return`, `render`, `jsx`, `tsx`
|
|
21
|
-
- Layout changes: `flex`, `grid`, `position`, `margin`, `padding`, `width`, `height`
|
|
22
|
-
- Visibility: `display`, `hidden`, `visible`, `opacity`
|
|
23
|
-
3. Cross-reference the `changedSectionsClassNames` from the diff summaries with class names
|
|
24
|
-
and component names in the PR diff to establish causal links.
|
|
25
|
-
|
|
26
|
-
## Output Format
|
|
27
|
-
|
|
28
|
-
Return a structured analysis under 800 words:
|
|
29
|
-
|
|
30
|
-
### Files Changed
|
|
31
|
-
|
|
32
|
-
List all changed files with +/- line counts, grouped by category:
|
|
33
|
-
|
|
34
|
-
- **Visual/Styling**: CSS, SCSS, component files with UI changes
|
|
35
|
-
- **Logic/Data**: API calls, state management, utilities
|
|
36
|
-
- **Config/Other**: Package files, configs, tests
|
|
37
|
-
|
|
38
|
-
### Code-to-Diff Correlation
|
|
39
|
-
|
|
40
|
-
For each screenshot that differs (from the diff summaries):
|
|
41
|
-
|
|
42
|
-
- Screenshot identifier and mismatch percentage
|
|
43
|
-
- The `changedSectionsClassNames` from the diff
|
|
44
|
-
- Which PR file(s) most likely caused this diff, with specific line references
|
|
45
|
-
- Whether this visual change appears **expected** (intentional, matches code intent) or
|
|
46
|
-
**unexpected** (no clear code change correspondence)
|
|
47
|
-
|
|
48
|
-
### Uncorrelated Changes
|
|
49
|
-
|
|
50
|
-
Note any PR changes that should affect visual output but don't appear in any screenshot
|
|
51
|
-
diffs (possible coverage gaps), and any screenshot diffs that don't correspond to any
|
|
52
|
-
obvious code change (possible flakes or indirect effects).
|
|
53
|
-
|
|
54
|
-
### Summary
|
|
55
|
-
|
|
56
|
-
One paragraph: what is this PR doing, which visual changes are expected, and which (if any)
|
|
57
|
-
warrant further investigation?
|
|
@@ -1,75 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: summarizer
|
|
3
|
-
description: Summarizes large files (logs, timelines, session data, diffs) that are too large to read in full. Use when a file exceeds 5000 lines or when you need a quick overview of a large file's contents.
|
|
4
|
-
tools: Read, Grep, Glob
|
|
5
|
-
model: haiku
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
You are a file summarization specialist for debugging Meticulous replay issues.
|
|
9
|
-
|
|
10
|
-
When given a file to summarize, produce a concise overview that helps the main agent
|
|
11
|
-
decide what to investigate further. Do not read the entire file -- use Grep and targeted
|
|
12
|
-
reads to extract the key information efficiently.
|
|
13
|
-
|
|
14
|
-
## Process
|
|
15
|
-
|
|
16
|
-
1. Read the first ~50 lines to understand the file's structure and format.
|
|
17
|
-
2. Use Grep to find key patterns: errors, warnings, screenshots, network failures,
|
|
18
|
-
timeouts, navigation events, and any terms the caller highlighted.
|
|
19
|
-
3. Read targeted sections around important matches.
|
|
20
|
-
4. Read the last ~30 lines for final state or summary information.
|
|
21
|
-
5. Produce a structured summary.
|
|
22
|
-
|
|
23
|
-
## File-Type Guidelines
|
|
24
|
-
|
|
25
|
-
### Log files (`logs.deterministic.txt`, `logs.deterministic.filtered.txt`, `logs.concise.txt`)
|
|
26
|
-
|
|
27
|
-
Summarize:
|
|
28
|
-
|
|
29
|
-
- Approximate line count and virtual time range
|
|
30
|
-
- Key phases: navigation, network loading, user events, screenshots
|
|
31
|
-
- Errors, warnings, or unusual patterns (grep for `error`, `warning`, `fail`, `timeout`)
|
|
32
|
-
- Network request overview: grep for `request` and note counts, failures
|
|
33
|
-
- Screenshot timestamps and event numbers
|
|
34
|
-
|
|
35
|
-
### Timeline files (`timeline.json`)
|
|
36
|
-
|
|
37
|
-
Summarize:
|
|
38
|
-
|
|
39
|
-
- Total entry count
|
|
40
|
-
- Event kind breakdown (grep for `"kind":` and tally)
|
|
41
|
-
- Any `potentialFlakinessWarning` entries
|
|
42
|
-
- Virtual time range (first and last entries)
|
|
43
|
-
- Notable gaps or clusters of events
|
|
44
|
-
|
|
45
|
-
### Session data (`sessions/*/data.json`)
|
|
46
|
-
|
|
47
|
-
Summarize:
|
|
48
|
-
|
|
49
|
-
- Session structure (grep for top-level keys)
|
|
50
|
-
- User interaction count and types
|
|
51
|
-
- Network request patterns (count, domains)
|
|
52
|
-
- Any storage or cookie data of note
|
|
53
|
-
|
|
54
|
-
### Diff files (`log-diffs/*.diff`, `log-diffs/*.filtered.diff`)
|
|
55
|
-
|
|
56
|
-
Summarize:
|
|
57
|
-
|
|
58
|
-
- Total hunks and changed line counts
|
|
59
|
-
- Categories of changes (network, animation, timers, navigation)
|
|
60
|
-
- Location of first divergence
|
|
61
|
-
- Whether changes are concentrated or spread throughout
|
|
62
|
-
|
|
63
|
-
### Any other file
|
|
64
|
-
|
|
65
|
-
Summarize:
|
|
66
|
-
|
|
67
|
-
- File structure and format
|
|
68
|
-
- Size and key sections
|
|
69
|
-
- Notable content relevant to debugging
|
|
70
|
-
|
|
71
|
-
## Output Format
|
|
72
|
-
|
|
73
|
-
Return a summary under 500 words. Include specific line numbers or grep patterns so
|
|
74
|
-
the main agent can follow up on anything interesting. Structure the summary with clear
|
|
75
|
-
headings for easy scanning.
|
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"permissions": {
|
|
3
|
-
"allow": [
|
|
4
|
-
"Read",
|
|
5
|
-
"Grep",
|
|
6
|
-
"Glob",
|
|
7
|
-
"Task",
|
|
8
|
-
"Bash(find *)",
|
|
9
|
-
"Bash(grep *)",
|
|
10
|
-
"Bash(ls *)",
|
|
11
|
-
"Bash(git diff *)",
|
|
12
|
-
"Bash(git log *)",
|
|
13
|
-
"Bash(git show *)",
|
|
14
|
-
"Bash(git status)",
|
|
15
|
-
"Bash(git rev-parse *)",
|
|
16
|
-
"Bash(git blame *)",
|
|
17
|
-
"Bash(git grep *)",
|
|
18
|
-
"Bash(diff *)",
|
|
19
|
-
"Bash(wc *)",
|
|
20
|
-
"Bash(head *)",
|
|
21
|
-
"Bash(tail *)",
|
|
22
|
-
"Bash(sort *)",
|
|
23
|
-
"Bash(uniq *)",
|
|
24
|
-
"Bash(jq *)",
|
|
25
|
-
"Bash(cat *)"
|
|
26
|
-
],
|
|
27
|
-
"ask": ["WebFetch", "WebSearch"],
|
|
28
|
-
"deny": [
|
|
29
|
-
"Edit",
|
|
30
|
-
"Write",
|
|
31
|
-
"mcp__*",
|
|
32
|
-
"ToolSearch",
|
|
33
|
-
"ListMcpResourcesTool",
|
|
34
|
-
"ReadMcpResourceTool"
|
|
35
|
-
],
|
|
36
|
-
"defaultMode": "default"
|
|
37
|
-
},
|
|
38
|
-
"spinnerVerbs": {
|
|
39
|
-
"mode": "replace",
|
|
40
|
-
"verbs": [
|
|
41
|
-
"Analyzing",
|
|
42
|
-
"Investigating",
|
|
43
|
-
"Comparing",
|
|
44
|
-
"Searching",
|
|
45
|
-
"Processing",
|
|
46
|
-
"Squinting at screenshots",
|
|
47
|
-
"Traversing tangled timelines",
|
|
48
|
-
"Fighting fickle flakes",
|
|
49
|
-
"Diagnosing dubious diffs",
|
|
50
|
-
"Reconciling replays",
|
|
51
|
-
"Deciphering devious divergences",
|
|
52
|
-
"Debugging flakes",
|
|
53
|
-
"Patching network sessions",
|
|
54
|
-
"Determining nondeterminism"
|
|
55
|
-
]
|
|
56
|
-
}
|
|
57
|
-
}
|
|
@@ -1,61 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: debugging-diffs
|
|
3
|
-
description: Investigate unexpected visual differences between head and base replays. Use when screenshot diffs are flagged or visual regressions are reported.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Debugging Screenshot Diffs
|
|
7
|
-
|
|
8
|
-
Use this guide when investigating unexpected visual differences between head and base replays.
|
|
9
|
-
|
|
10
|
-
## Investigation Steps
|
|
11
|
-
|
|
12
|
-
### 1. Understand the Diffs
|
|
13
|
-
|
|
14
|
-
- Read the replay diff JSON in `debug-data/diffs/<id>.json`.
|
|
15
|
-
- Check `screenshotDiffResults` for which screenshots differ.
|
|
16
|
-
- Note the diff percentage and pixel count for each screenshot.
|
|
17
|
-
|
|
18
|
-
### 2. Correlate with Code Changes
|
|
19
|
-
|
|
20
|
-
- Check `commitSha` in `context.json` to identify the code changes.
|
|
21
|
-
- If `project-repo/` is available, use `git log` and `git diff` to see what changed.
|
|
22
|
-
- Focus on CSS changes, component rendering logic, and layout modifications.
|
|
23
|
-
|
|
24
|
-
### 3. Compare Logs at Screenshot Time
|
|
25
|
-
|
|
26
|
-
- Find the screenshot timestamps in `timeline.json`.
|
|
27
|
-
- Compare what events occurred before each screenshot in head vs base.
|
|
28
|
-
- Look for missing or extra events that could cause visual differences.
|
|
29
|
-
|
|
30
|
-
### 4. Check for Expected vs Unexpected Diffs
|
|
31
|
-
|
|
32
|
-
- **Expected**: Code changes that intentionally modify the UI (new features, style updates).
|
|
33
|
-
- **Unexpected**: Same code producing different visual output, or unrelated areas changing.
|
|
34
|
-
- Check if the diff is in a dynamic content area (timestamps, counters, user-specific data).
|
|
35
|
-
|
|
36
|
-
<!-- if-snapshot-assets -->
|
|
37
|
-
|
|
38
|
-
### 5. Examine Snapshotted Assets
|
|
39
|
-
|
|
40
|
-
- Compare JS/CSS between head and base under `debug-data/replays/{head,base}/<replayId>/snapshotted-assets/`.
|
|
41
|
-
- Look for changes in CSS that could cause layout shifts.
|
|
42
|
-
- Check for new or modified JavaScript that affects rendering.
|
|
43
|
-
|
|
44
|
-
<!-- end-if-snapshot-assets -->
|
|
45
|
-
|
|
46
|
-
### 6. Review Screenshot Assertions Config
|
|
47
|
-
|
|
48
|
-
- Check `screenshotAssertionsOptions` in the diff JSON for threshold settings.
|
|
49
|
-
- Some diffs may be within acceptable tolerance but still flagged.
|
|
50
|
-
|
|
51
|
-
### 7. Known Meticulous Replay Behaviors (Safe to Approve)
|
|
52
|
-
|
|
53
|
-
The following patterns are caused by Meticulous replay engine behavior, not by your code changes. When these are the root cause of a diff, the diff is safe to approve.
|
|
54
|
-
|
|
55
|
-
**Animation flakes**: Continuous animations (Lottie, CSS `@keyframes`, `requestAnimationFrame` loops) can cause screenshots to land at slightly different animation states between base and head, even with no code changes. Meticulous's DOM stability detection triggers extra animation frames trying to reach a stable state, but continuous animations never fully stabilize. **How to confirm**: check `replayComparison` in `context.json` for differing `totalAnimationFrames`; check log diffs for different animation frame counts; search the codebase for `lottie`, `requestAnimationFrame`, CSS `animation`, `@keyframes`, `<canvas>`, `<video>`. If the only visual difference is an animation in a different frame, this diff is safe to approve.
|
|
56
|
-
|
|
57
|
-
**Network timing differences**: When multiple network responses complete at the same virtual time, slight ordering differences can cause minor rendering variations (e.g. a list rendering in a different order before settling). **How to confirm**: log diffs show network requests completing in a different order at the same virtual time, but the final rendered state is the same or nearly identical. If no code change caused the reordering, this diff is safe to approve.
|
|
58
|
-
|
|
59
|
-
**Extra DOM stability frames**: The head replay may show more timeline events before a screenshot than the base, caused by DOM stability detection running additional animation frames. If the extra events are all animation-frame-related and the visual diff is minor, this diff is safe to approve.
|
|
60
|
-
|
|
61
|
-
When any of these patterns is the root cause, tell the user: "This diff is caused by a known Meticulous replay behavior, not by your code changes. It is safe to approve."
|
|
@@ -1,52 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: debugging-flakes
|
|
3
|
-
description: Investigate flaky or non-deterministic replay behavior. Use when replays produce different results across runs of the same session against the same code.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Debugging Flaky Replays
|
|
7
|
-
|
|
8
|
-
A flaky replay is one that produces different results (screenshots, logs, behavior) across
|
|
9
|
-
multiple runs of the same session against the same code. Use this guide when investigating
|
|
10
|
-
non-deterministic behavior.
|
|
11
|
-
|
|
12
|
-
## Investigation Steps
|
|
13
|
-
|
|
14
|
-
### 1. Compare Deterministic Logs
|
|
15
|
-
|
|
16
|
-
- Diff `logs.deterministic.txt` between the head and base replays.
|
|
17
|
-
- Look for the first point of divergence -- this is usually where the flake originates.
|
|
18
|
-
- Pay attention to event execution order differences.
|
|
19
|
-
|
|
20
|
-
### 2. Check Timeline for Timing Issues
|
|
21
|
-
|
|
22
|
-
- Read `timeline.json` and look for events with significantly different virtual timestamps.
|
|
23
|
-
- Look for race conditions: events that depend on network responses or animations completing.
|
|
24
|
-
- Check for `setTimeout`/`setInterval` patterns that may resolve differently.
|
|
25
|
-
|
|
26
|
-
### 3. Analyze Accuracy Data
|
|
27
|
-
|
|
28
|
-
- Read `accuracyData.json` for the replay's self-assessment of accuracy.
|
|
29
|
-
- Low accuracy scores often correlate with flakiness.
|
|
30
|
-
- Check which specific checks failed.
|
|
31
|
-
|
|
32
|
-
### 4. Look for Common Flake Patterns
|
|
33
|
-
|
|
34
|
-
- **Animation timing**: CSS transitions or JS animations that haven't completed when screenshot is taken.
|
|
35
|
-
- **Network race conditions**: API responses arriving in different order.
|
|
36
|
-
- **Date/time dependencies**: Code that uses `Date.now()` or renders timestamps.
|
|
37
|
-
- **Random values**: Code using `Math.random()` that wasn't properly seeded.
|
|
38
|
-
- **Lazy loading**: Content loaded on intersection observer timing.
|
|
39
|
-
- **Font loading**: Web fonts loading at different times causing layout shifts.
|
|
40
|
-
|
|
41
|
-
When any of these patterns is identified as the root cause and no user code change is involved, the resulting diff is caused by non-determinism in the replay environment, not the user's code. Advise the user that these diffs are **safe to approve**.
|
|
42
|
-
|
|
43
|
-
### 5. Check Replay Parameters
|
|
44
|
-
|
|
45
|
-
- Compare `launchBrowserAndReplayParams.json` between runs.
|
|
46
|
-
- Verify viewport size, user agent, and other environment settings match.
|
|
47
|
-
- Check if network stubbing configuration is consistent.
|
|
48
|
-
|
|
49
|
-
### 6. Examine Stack Traces
|
|
50
|
-
|
|
51
|
-
- Read `stackTraces.json` for any errors thrown during replay.
|
|
52
|
-
- Errors that occur in some runs but not others are strong flake indicators.
|
|
@@ -1,45 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: debugging-network
|
|
3
|
-
description: Investigate network-related replay failures and divergences. Use when replays fail due to network errors, stubbing issues, or request ordering problems.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Debugging Network Issues
|
|
7
|
-
|
|
8
|
-
Use this guide when replays fail or diverge due to network request problems.
|
|
9
|
-
|
|
10
|
-
## Investigation Steps
|
|
11
|
-
|
|
12
|
-
### 1. Check Logs for Network Errors
|
|
13
|
-
|
|
14
|
-
- Search `logs.concise.txt` for "network", "fetch", "xhr", "request", "response", "timeout".
|
|
15
|
-
- Look for failed requests, unexpected status codes, or missing responses.
|
|
16
|
-
- Check for CORS errors or SSL issues.
|
|
17
|
-
|
|
18
|
-
### 2. Compare Network Activity in Timeline
|
|
19
|
-
|
|
20
|
-
- In `timeline.json`, look for network-related events.
|
|
21
|
-
- Compare the sequence and timing of network requests between head and base.
|
|
22
|
-
- Look for requests in one replay that are missing in the other.
|
|
23
|
-
|
|
24
|
-
### 3. Examine Session Data
|
|
25
|
-
|
|
26
|
-
- Read session data in `debug-data/sessions/<id>/data.json`.
|
|
27
|
-
- Check `recordedRequests` for the original HAR entries captured during recording.
|
|
28
|
-
- Compare recorded requests with what was replayed.
|
|
29
|
-
|
|
30
|
-
### 4. Look for Stubbing Issues
|
|
31
|
-
|
|
32
|
-
- Network requests are stubbed during replay using recorded data.
|
|
33
|
-
- Check if new API endpoints were added that don't have recorded responses.
|
|
34
|
-
- Look for requests with dynamic parameters (timestamps, tokens) that may not match stubs.
|
|
35
|
-
|
|
36
|
-
### 5. Check for Request Ordering Dependencies
|
|
37
|
-
|
|
38
|
-
- Some applications depend on requests completing in a specific order.
|
|
39
|
-
- Look for race conditions where parallel requests resolve differently.
|
|
40
|
-
- Check for waterfall dependencies (request B depends on response from request A).
|
|
41
|
-
|
|
42
|
-
### 6. Verify API Compatibility
|
|
43
|
-
|
|
44
|
-
- If the API was changed, recorded responses may no longer be valid.
|
|
45
|
-
- Check for schema changes, new required fields, or renamed endpoints.
|
|
@@ -1,47 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: debugging-sessions
|
|
3
|
-
description: Investigate problems with recorded session data. Use when session recordings appear incomplete, corrupted, or contain unexpected data.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Debugging Session Data Issues
|
|
7
|
-
|
|
8
|
-
Use this guide when investigating problems with the recorded session data itself.
|
|
9
|
-
|
|
10
|
-
## Investigation Steps
|
|
11
|
-
|
|
12
|
-
### 1. Examine Session Structure
|
|
13
|
-
|
|
14
|
-
- Read `debug-data/sessions/<id>/data.json` (this can be very large, use grep/search).
|
|
15
|
-
- Key fields: `rrwebEvents`, `userEvents`, `recordedRequests`, `applicationStorage`, `webSockets`.
|
|
16
|
-
|
|
17
|
-
### 2. Check User Events
|
|
18
|
-
|
|
19
|
-
- `userEvents` contains the sequence of user interactions that will be replayed.
|
|
20
|
-
- Verify events are in chronological order.
|
|
21
|
-
- Check for truncated or incomplete event sequences.
|
|
22
|
-
- Look for unusually rapid event sequences that may indicate automated behavior.
|
|
23
|
-
|
|
24
|
-
### 3. Verify Network Recordings
|
|
25
|
-
|
|
26
|
-
- `recordedRequests` contains HAR-format entries of network activity.
|
|
27
|
-
- Check for missing responses (the request was recorded but the response wasn't).
|
|
28
|
-
- Look for very large responses that might have been truncated.
|
|
29
|
-
- Verify content types and encoding are preserved correctly.
|
|
30
|
-
|
|
31
|
-
### 4. Check Application Storage
|
|
32
|
-
|
|
33
|
-
- `applicationStorage` captures localStorage, sessionStorage, and cookies.
|
|
34
|
-
- Verify that authentication state is properly captured.
|
|
35
|
-
- Look for expired tokens or sessions that may cause different behavior during replay.
|
|
36
|
-
|
|
37
|
-
### 5. Look for Session Quality Issues
|
|
38
|
-
|
|
39
|
-
- Very short sessions (few events) may not provide meaningful coverage.
|
|
40
|
-
- Sessions with `abandoned: true` were not completed normally.
|
|
41
|
-
- Check `numberUserEvents` and `numberBytes` for unusually small or large values.
|
|
42
|
-
|
|
43
|
-
### 6. Verify Recording Environment
|
|
44
|
-
|
|
45
|
-
- Check session metadata for the recording environment (hostname, URL).
|
|
46
|
-
- Ensure the session was recorded against a compatible version of the application.
|
|
47
|
-
- Look for environment-specific behavior (staging vs production data).
|
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: debugging-timelines
|
|
3
|
-
description: Investigate timeline divergence between head and base replays. Use when event sequences, ordering, or timing differ unexpectedly.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Debugging Timeline Divergence
|
|
7
|
-
|
|
8
|
-
Use this guide when replay timelines differ unexpectedly between head and base runs.
|
|
9
|
-
|
|
10
|
-
## Investigation Steps
|
|
11
|
-
|
|
12
|
-
### 1. Load and Compare Timelines
|
|
13
|
-
|
|
14
|
-
- Read `timeline.json` from both head and base replay directories.
|
|
15
|
-
- The timeline is an array of events with timestamps, types, and data.
|
|
16
|
-
- Look for the first event where the timelines diverge.
|
|
17
|
-
|
|
18
|
-
### 2. Understand Event Types
|
|
19
|
-
|
|
20
|
-
Key timeline event types:
|
|
21
|
-
|
|
22
|
-
- **user-event**: User interactions (click, type, scroll, hover).
|
|
23
|
-
- **network-request**: API calls and responses.
|
|
24
|
-
- **screenshot**: Screenshot capture points.
|
|
25
|
-
- **mutation**: DOM mutations observed during replay.
|
|
26
|
-
- **navigation**: Page navigation events.
|
|
27
|
-
- **error**: JavaScript errors.
|
|
28
|
-
- **console**: Console log messages.
|
|
29
|
-
|
|
30
|
-
### 3. Identify Divergence Patterns
|
|
31
|
-
|
|
32
|
-
- **Missing events**: Events in base that don't appear in head (or vice versa).
|
|
33
|
-
- **Reordered events**: Same events but in different sequence.
|
|
34
|
-
- **Timing shifts**: Events at significantly different virtual timestamps.
|
|
35
|
-
- **Extra events**: New events not present in the baseline.
|
|
36
|
-
|
|
37
|
-
### 4. Check Timeline Stats
|
|
38
|
-
|
|
39
|
-
- Read `timeline-stats.json` for aggregated statistics.
|
|
40
|
-
- Compare event counts, durations, and error counts between replays.
|
|
41
|
-
|
|
42
|
-
### 5. Trace Back to Root Cause
|
|
43
|
-
|
|
44
|
-
- Once you find the divergence point, look at what happened immediately before.
|
|
45
|
-
- Check if a user event triggered different behavior.
|
|
46
|
-
- Look for conditional logic in the application that might execute differently.
|
|
47
|
-
|
|
48
|
-
### 6. Cross-Reference with Logs
|
|
49
|
-
|
|
50
|
-
- Use timestamps from the timeline divergence to find corresponding log entries.
|
|
51
|
-
- Check `logs.deterministic.txt` at the same virtual time for additional context.
|
|
@@ -1,20 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: pr-analysis
|
|
3
|
-
description: Analyze PR source code changes and correlate with screenshot diffs. Use when pr-diff.txt is present and you need to understand which code changes caused visual differences.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# PR Analysis
|
|
7
|
-
|
|
8
|
-
When `debug-data/pr-diff.txt` is present in the workspace, analyze the source code changes and correlate
|
|
9
|
-
them with the screenshot diffs.
|
|
10
|
-
|
|
11
|
-
1. Read `debug-data/pr-diff.txt` to understand what code changed
|
|
12
|
-
2. Read the diff summaries in `debug-data/diffs/*.summary.json` to see which screenshots differ
|
|
13
|
-
3. For each screenshot that differs, identify which code changes are most likely responsible
|
|
14
|
-
|
|
15
|
-
Provide a structured analysis:
|
|
16
|
-
|
|
17
|
-
- Which files were modified and what the key changes are
|
|
18
|
-
- Which code changes are most likely to affect visual output (CSS, layout, component rendering)
|
|
19
|
-
- For each differing screenshot, the most likely code change that caused it
|
|
20
|
-
- Whether the visual changes appear intentional (matching the code intent) or unintentional
|