@vizzly-testing/cli 0.11.0 → 0.11.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/claude-plugin/.claude-plugin/README.md +266 -0
- package/claude-plugin/CHANGELOG.md +58 -0
- package/{.claude-plugin → claude-plugin}/mcp/vizzly-server/cloud-api-provider.js +29 -0
- package/{.claude-plugin → claude-plugin}/mcp/vizzly-server/index.js +54 -1
- package/claude-plugin/skills/check-visual-tests/SKILL.md +158 -0
- package/claude-plugin/skills/debug-visual-regression/SKILL.md +269 -0
- package/dist/plugin-loader.js +6 -14
- package/dist/services/api-service.js +31 -1
- package/dist/services/tdd-service.js +32 -5
- package/dist/types/services/api-service.d.ts +14 -0
- package/package.json +2 -2
- package/.claude-plugin/README.md +0 -114
- package/.claude-plugin/commands/debug-diff.md +0 -153
- package/.claude-plugin/commands/tdd-status.md +0 -43
- /package/{.claude-plugin → claude-plugin/.claude-plugin}/marketplace.json +0 -0
- /package/{.claude-plugin → claude-plugin/.claude-plugin}/plugin.json +0 -0
- /package/{.claude-plugin → claude-plugin}/.mcp.json +0 -0
- /package/{.claude-plugin → claude-plugin}/commands/setup.md +0 -0
- /package/{.claude-plugin → claude-plugin}/commands/suggest-screenshots.md +0 -0
- /package/{.claude-plugin → claude-plugin}/mcp/vizzly-server/local-tdd-provider.js +0 -0
- /package/{.claude-plugin → claude-plugin}/mcp/vizzly-server/token-resolver.js +0 -0
|
@@ -1,153 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
description: Analyze a specific visual regression failure and suggest fixes
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
# Debug Vizzly Visual Regression
|
|
6
|
-
|
|
7
|
-
Analyze a specific failing visual comparison in detail. Supports both local TDD mode and cloud API mode.
|
|
8
|
-
|
|
9
|
-
## Process
|
|
10
|
-
|
|
11
|
-
1. **Call the unified MCP tool**: Use `read_comparison_details` with the identifier (screenshot name or comparison ID)
|
|
12
|
-
- The tool automatically detects whether to use local TDD mode or cloud mode
|
|
13
|
-
- Pass screenshot name (e.g., "homepage_desktop") for local mode
|
|
14
|
-
- Pass comparison ID (e.g., "cmp_abc123") for cloud mode
|
|
15
|
-
- Returns a response with `mode` field indicating which mode was used
|
|
16
|
-
|
|
17
|
-
2. **Check the mode in the response**:
|
|
18
|
-
- **Local mode** (`mode: "local"`): Returns filesystem paths (`baselinePath`, `currentPath`, `diffPath`)
|
|
19
|
-
- **Cloud mode** (`mode: "cloud"`): Returns URLs (`baselineUrl`, `currentUrl`, `diffUrl`)
|
|
20
|
-
|
|
21
|
-
3. **Analyze the comparison data**:
|
|
22
|
-
- Diff percentage and threshold
|
|
23
|
-
- Status (failed/new/passed)
|
|
24
|
-
- Image references (paths or URLs depending on mode)
|
|
25
|
-
|
|
26
|
-
4. **View the actual images** (critical for visual analysis):
|
|
27
|
-
|
|
28
|
-
Check the `mode` field in the response to determine which tool to use:
|
|
29
|
-
|
|
30
|
-
**If mode is "local":**
|
|
31
|
-
- Response contains filesystem paths (`baselinePath`, `currentPath`, `diffPath`)
|
|
32
|
-
- **Use the Read tool to view ONLY baselinePath and currentPath**
|
|
33
|
-
- **DO NOT read diffPath** - it causes API errors
|
|
34
|
-
|
|
35
|
-
**If mode is "cloud":**
|
|
36
|
-
- Response contains public URLs (`baselineUrl`, `currentUrl`, `diffUrl`)
|
|
37
|
-
- **Use the WebFetch tool to view ONLY baselineUrl and currentUrl**
|
|
38
|
-
- **DO NOT fetch diffUrl** - it causes API errors
|
|
39
|
-
|
|
40
|
-
**IMPORTANT:** You MUST view the baseline and current images to provide accurate analysis
|
|
41
|
-
|
|
42
|
-
5. **Provide detailed visual insights** based on what you see:
|
|
43
|
-
- What type of change was detected (small/moderate/large diff)
|
|
44
|
-
- Describe the specific visual differences you observe in the images
|
|
45
|
-
- Identify which UI components, elements, or layouts changed
|
|
46
|
-
- Possible causes based on diff percentage and visual inspection:
|
|
47
|
-
- <1%: Anti-aliasing, font rendering, subpixel differences
|
|
48
|
-
- 1-5%: Layout shifts, padding/margin changes, color variations
|
|
49
|
-
- > 5%: Significant layout changes, missing content, major visual updates
|
|
50
|
-
|
|
51
|
-
6. **Suggest next steps** based on the mode:
|
|
52
|
-
- **If local mode**: Whether to accept using `accept_baseline` tool
|
|
53
|
-
- **If cloud mode**: Whether to approve/reject using `approve_comparison` or `reject_comparison` tools
|
|
54
|
-
- Areas to investigate if unintentional
|
|
55
|
-
- How to fix common issues
|
|
56
|
-
- Specific code changes if you can identify them from the visual diff
|
|
57
|
-
|
|
58
|
-
## Example Analysis (Local TDD Mode)
|
|
59
|
-
|
|
60
|
-
```
|
|
61
|
-
Step 1: Call read_comparison_details with screenshot name
|
|
62
|
-
Tool: read_comparison_details({ identifier: "homepage" })
|
|
63
|
-
|
|
64
|
-
Response:
|
|
65
|
-
{
|
|
66
|
-
"name": "homepage",
|
|
67
|
-
"status": "failed",
|
|
68
|
-
"diffPercentage": 2.3,
|
|
69
|
-
"threshold": 0.1,
|
|
70
|
-
"mode": "local",
|
|
71
|
-
"baselinePath": "/Users/you/project/.vizzly/baselines/homepage.png",
|
|
72
|
-
"currentPath": "/Users/you/project/.vizzly/screenshots/homepage.png",
|
|
73
|
-
"diffPath": "/Users/you/project/.vizzly/diffs/homepage.png"
|
|
74
|
-
}
|
|
75
|
-
|
|
76
|
-
Step 2: Detected mode is "local", so use Read tool for images
|
|
77
|
-
Read(baselinePath) and Read(currentPath)
|
|
78
|
-
|
|
79
|
-
Visual Analysis:
|
|
80
|
-
[After reading the baseline and current image files...]
|
|
81
|
-
|
|
82
|
-
Comparing the two images, the navigation header has shifted down by approximately 10-15 pixels.
|
|
83
|
-
Specific changes observed:
|
|
84
|
-
- The logo position moved from y:20px to y:35px
|
|
85
|
-
- Navigation menu items are now overlapping with the hero section
|
|
86
|
-
- The "Sign Up" button background changed from blue (#2563eb) to a darker blue (#1e40af)
|
|
87
|
-
|
|
88
|
-
Root Cause:
|
|
89
|
-
Based on the visual comparison, this appears to be a margin or padding change on the
|
|
90
|
-
header element. The button color change is likely a hover state being captured.
|
|
91
|
-
|
|
92
|
-
Recommendations:
|
|
93
|
-
1. Check for recent CSS changes to:
|
|
94
|
-
- `.header` or `nav` margin-top/padding-top
|
|
95
|
-
- Any global layout shifts affecting the header
|
|
96
|
-
2. The button color change suggests a hover state - ensure consistent state during screenshot capture
|
|
97
|
-
3. If the header position change is intentional:
|
|
98
|
-
- Accept as new baseline using `accept_baseline` tool
|
|
99
|
-
4. If unintentional:
|
|
100
|
-
- Revert CSS changes to header positioning
|
|
101
|
-
- Verify with: `git diff src/styles/header.css`
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
## Example Analysis (Cloud Mode)
|
|
105
|
-
|
|
106
|
-
```
|
|
107
|
-
Step 1: Call read_comparison_details with comparison ID
|
|
108
|
-
Tool: read_comparison_details({
|
|
109
|
-
identifier: "cmp_xyz789",
|
|
110
|
-
apiToken: "vzt_..."
|
|
111
|
-
})
|
|
112
|
-
|
|
113
|
-
Response:
|
|
114
|
-
{
|
|
115
|
-
"name": "homepage",
|
|
116
|
-
"status": "failed",
|
|
117
|
-
"diffPercentage": 2.3,
|
|
118
|
-
"threshold": 0.1,
|
|
119
|
-
"mode": "cloud",
|
|
120
|
-
"baselineUrl": "https://app.vizzly.dev/screenshots/abc123/baseline.png",
|
|
121
|
-
"currentUrl": "https://app.vizzly.dev/screenshots/abc123/current.png",
|
|
122
|
-
"diffUrl": "https://app.vizzly.dev/screenshots/abc123/diff.png",
|
|
123
|
-
"comparisonId": "cmp_xyz789",
|
|
124
|
-
"buildId": "bld_abc123"
|
|
125
|
-
}
|
|
126
|
-
|
|
127
|
-
Step 2: Detected mode is "cloud", so use WebFetch tool for images
|
|
128
|
-
WebFetch(baselineUrl) and WebFetch(currentUrl)
|
|
129
|
-
|
|
130
|
-
Visual Analysis:
|
|
131
|
-
[After fetching the baseline and current image URLs...]
|
|
132
|
-
|
|
133
|
-
[Same analysis as local mode example...]
|
|
134
|
-
|
|
135
|
-
Recommendations:
|
|
136
|
-
1. [Same technical recommendations as local mode...]
|
|
137
|
-
2. If the header position change is intentional:
|
|
138
|
-
- Approve this comparison using `approve_comparison` tool
|
|
139
|
-
3. If unintentional:
|
|
140
|
-
- Reject using `reject_comparison` tool with detailed reason
|
|
141
|
-
- Have the team fix the CSS changes
|
|
142
|
-
```
|
|
143
|
-
|
|
144
|
-
## Important Notes
|
|
145
|
-
|
|
146
|
-
- **Unified Tool**: Always use `read_comparison_details` with the identifier - it automatically detects the mode
|
|
147
|
-
- **Mode Detection**: Check the `mode` field in the response to know which viewing tool to use
|
|
148
|
-
- **Image Viewing**:
|
|
149
|
-
- Local mode → Use Read tool with filesystem paths
|
|
150
|
-
- Cloud mode → Use WebFetch tool with URLs
|
|
151
|
-
- **Diff Images**: NEVER attempt to view/read/fetch the diff image - it causes API errors
|
|
152
|
-
- **Visual Analysis**: Always view the baseline and current images before providing analysis
|
|
153
|
-
- Visual inspection reveals details that diff percentages alone cannot convey
|
|
@@ -1,43 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
description: Check TDD dashboard status and view visual regression test results
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
# Check Vizzly TDD Status
|
|
6
|
-
|
|
7
|
-
Use the Vizzly MCP server to check the current TDD status:
|
|
8
|
-
|
|
9
|
-
1. Call the `get_tdd_status` tool from the vizzly MCP server
|
|
10
|
-
2. Analyze the comparison results
|
|
11
|
-
3. Show a summary of:
|
|
12
|
-
- Total screenshots tested
|
|
13
|
-
- Passed, failed, and new screenshot counts
|
|
14
|
-
- List of failed comparisons with diff percentages
|
|
15
|
-
- Available diff images to inspect
|
|
16
|
-
4. If TDD server is running, provide the dashboard URL
|
|
17
|
-
5. For failed comparisons, provide guidance on next steps
|
|
18
|
-
|
|
19
|
-
## Example Output Format
|
|
20
|
-
|
|
21
|
-
```
|
|
22
|
-
Vizzly TDD Status:
|
|
23
|
-
✅ Total: 15 screenshots
|
|
24
|
-
✅ Passed: 12
|
|
25
|
-
❌ Failed: 2 (exceeded threshold)
|
|
26
|
-
🆕 New: 1 (no baseline)
|
|
27
|
-
|
|
28
|
-
Failed Comparisons:
|
|
29
|
-
- homepage (2.3% diff) - Check .vizzly/diffs/homepage.png
|
|
30
|
-
- login-form (1.8% diff) - Check .vizzly/diffs/login-form.png
|
|
31
|
-
|
|
32
|
-
New Screenshots:
|
|
33
|
-
- dashboard (no baseline for comparison)
|
|
34
|
-
|
|
35
|
-
Dashboard: http://localhost:47392
|
|
36
|
-
|
|
37
|
-
Next Steps:
|
|
38
|
-
- Review diff images to understand what changed
|
|
39
|
-
- Accept baselines from dashboard if changes are intentional
|
|
40
|
-
- Fix visual issues if changes are unintentional
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
Focus on providing actionable information to help the developer understand what's failing and why.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|