@vizzly-testing/cli 0.38.1 → 0.39.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +222 -195
- package/dist/cli.js +55 -54
- package/dist/commands/init.js +154 -483
- package/dist/commands/project.js +128 -44
- package/dist/commands/run.js +2 -2
- package/dist/commands/tdd.js +89 -38
- package/dist/commands/upload.js +2 -2
- package/dist/init/agents.js +140 -0
- package/dist/init/core.js +31 -0
- package/dist/init/prompts.js +44 -0
- package/dist/reporter/reporter-bundle.css +1 -1
- package/dist/reporter/reporter-bundle.iife.js +30 -61
- package/dist/reporter-ssr/ssr-entry.js +3 -6
- package/dist/server/handlers/tdd-handler.js +33 -2
- package/dist/services/index.js +8 -2
- package/dist/tdd/services/comparison-service.js +4 -1
- package/dist/tdd/tdd-service.js +14 -2
- package/dist/test-runner/core.js +28 -3
- package/dist/test-runner/operations.js +41 -20
- package/dist/types/index.d.ts +8 -0
- package/dist/uploader/core.js +1 -0
- package/dist/utils/cli-update.js +105 -0
- package/dist/utils/config-discovery.js +73 -0
- package/dist/utils/config-loader.js +29 -8
- package/dist/utils/config-schema.js +5 -0
- package/dist/utils/file-changes.js +105 -0
- package/dist/utils/project-binding.js +96 -0
- package/dist/utils/project-link-store.js +13 -7
- package/dist/utils/update-check.js +2 -0
- package/dist/utils/update-notifications.js +175 -0
- package/package.json +16 -16
- package/skills/vizzly/SKILL.md +4 -3
- package/skills/vizzly/references/ci.md +33 -0
- package/skills/vizzly/references/cli-context.md +81 -71
- package/skills/vizzly/references/dynamic-content.md +16 -17
- package/skills/vizzly/references/sdks.md +22 -22
- package/skills/vizzly/references/setup-ci.md +0 -44
|
@@ -1,12 +1,12 @@
|
|
|
1
|
-
# CLI
|
|
1
|
+
# CLI context
|
|
2
2
|
|
|
3
|
-
Use the
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
Use the repo's usual CLI command and existing credentials. If cloud auth isn't
|
|
4
|
+
available, report it. Only start an interactive login when setup is part of
|
|
5
|
+
the task.
|
|
6
6
|
|
|
7
|
-
## Query
|
|
7
|
+
## Query the cloud API
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Check the live API schema for endpoints and options:
|
|
10
10
|
|
|
11
11
|
```bash
|
|
12
12
|
vizzly api schema --json
|
|
@@ -15,34 +15,38 @@ vizzly api schema <operation-id> -q view=fields --json
|
|
|
15
15
|
vizzly api schema <operation-id> -q view=response --json
|
|
16
16
|
```
|
|
17
17
|
|
|
18
|
-
The index lists available
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
`data.response`.
|
|
18
|
+
The index lists available requests. Each operation shows the method, path,
|
|
19
|
+
parameters, auth, and example arguments. Ask for field choices or response types
|
|
20
|
+
as needed. API results are under `data.response`.
|
|
22
21
|
|
|
23
|
-
Call the
|
|
24
|
-
for headers, and `-q` for query parameters.
|
|
25
|
-
|
|
26
|
-
Follow the response's pagination values explicitly; the CLI fetches one page
|
|
27
|
-
per request. Keep cursors opaque and preserve the query they belong to.
|
|
22
|
+
Call the returned path with `vizzly api <path>`. Use `-X` for the method, `-H`
|
|
23
|
+
for headers, and `-q` for query parameters. Include the API version header from
|
|
24
|
+
the schema on data requests. Use `fields` to ask for just the data you need.
|
|
28
25
|
|
|
29
|
-
|
|
26
|
+
The CLI fetches one page per request. Use the response's page fields to get
|
|
27
|
+
more results. Pass cursors back unchanged and keep the same query.
|
|
28
|
+
|
|
29
|
+
To review a build, find the project and build, then check its screenshots,
|
|
30
30
|
comparisons, and image endpoints. Download images with `--output <new-file>`
|
|
31
|
-
and view baseline, current, and diff together.
|
|
32
|
-
|
|
33
|
-
when needed:
|
|
31
|
+
and view the baseline, current, and diff together. File output also helps with
|
|
32
|
+
large analysis results. Existing files won't be overwritten. Export the full
|
|
33
|
+
schema only when needed:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
vizzly api schema --full --output <new-file> --json
|
|
37
|
+
```
|
|
34
38
|
|
|
35
|
-
Review decisions
|
|
36
|
-
organization header.
|
|
37
|
-
with `-d @<file>` or `-d @-` for stdin.
|
|
38
|
-
intended decision, then read back the review state.
|
|
39
|
-
automatically
|
|
40
|
-
Treat schema examples as argument arrays, not shell scripts.
|
|
39
|
+
Review decisions need the user's authorization, user credentials, and the
|
|
40
|
+
organization header listed in the schema. Check the decision request's body
|
|
41
|
+
before sending it with `-d @<file>` or `-d @-` for stdin. Make a fresh `commandId`
|
|
42
|
+
for each intended decision, then read back the review state. Writes aren't
|
|
43
|
+
retried automatically. If you're unsure whether one succeeded, check the state
|
|
44
|
+
before trying again. Treat schema examples as argument arrays, not shell scripts.
|
|
41
45
|
|
|
42
|
-
## Choose
|
|
46
|
+
## Choose the evidence
|
|
43
47
|
|
|
44
|
-
Use
|
|
45
|
-
|
|
48
|
+
Use the ID supplied by the task. If no cloud build is named, find recent builds
|
|
49
|
+
and choose the one matching the branch, commit, or pull request:
|
|
46
50
|
|
|
47
51
|
```bash
|
|
48
52
|
vizzly builds --branch <branch> --limit 5 --json
|
|
@@ -50,12 +54,11 @@ vizzly status <build-id> --json
|
|
|
50
54
|
vizzly api schema sdk.getBuildContext --json
|
|
51
55
|
```
|
|
52
56
|
|
|
53
|
-
Use status
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
task.
|
|
57
|
+
Use status to check build progress, then follow the build-context API for
|
|
58
|
+
screenshots and diffs. Keep the API's comparison order and inspect the records
|
|
59
|
+
that matter to the task. The first comparison isn't necessarily the most useful.
|
|
57
60
|
|
|
58
|
-
For saved local
|
|
61
|
+
For saved local results:
|
|
59
62
|
|
|
60
63
|
```bash
|
|
61
64
|
vizzly context build current --source local --json
|
|
@@ -63,18 +66,18 @@ vizzly context screenshot "<screenshot-name>" --source local --json
|
|
|
63
66
|
vizzly context review-queue --source local --json
|
|
64
67
|
```
|
|
65
68
|
|
|
66
|
-
|
|
69
|
+
Check that the saved build, branch, timestamp, and baseline are recent enough
|
|
67
70
|
for the task.
|
|
68
71
|
|
|
69
|
-
## Generate
|
|
72
|
+
## Generate fresh evidence
|
|
70
73
|
|
|
71
|
-
For one run, let Vizzly
|
|
74
|
+
For one run, let Vizzly start and stop the local server:
|
|
72
75
|
|
|
73
76
|
```bash
|
|
74
77
|
vizzly tdd run "<existing visual test command>" --no-open
|
|
75
78
|
```
|
|
76
79
|
|
|
77
|
-
For repeated runs, start
|
|
80
|
+
For repeated runs, start a background server once:
|
|
78
81
|
|
|
79
82
|
```bash
|
|
80
83
|
vizzly tdd start --json
|
|
@@ -83,13 +86,12 @@ vizzly tdd status --json
|
|
|
83
86
|
vizzly tdd stop --json
|
|
84
87
|
```
|
|
85
88
|
|
|
86
|
-
`tdd run`
|
|
87
|
-
current task.
|
|
89
|
+
Choose `tdd run` or `tdd start`. Only stop a server started for this task.
|
|
88
90
|
|
|
89
|
-
## Review
|
|
91
|
+
## Review a local TDD run
|
|
90
92
|
|
|
91
|
-
After `tdd run` exits,
|
|
92
|
-
|
|
93
|
+
After `tdd run` exits, read captures from `.vizzly/`. You don't need a running
|
|
94
|
+
server or cloud credentials:
|
|
93
95
|
|
|
94
96
|
```bash
|
|
95
97
|
vizzly tdd screenshots list --page 1 --page-size 20
|
|
@@ -97,49 +99,61 @@ vizzly tdd screenshots latest
|
|
|
97
99
|
vizzly tdd screenshots show <comparison-id>
|
|
98
100
|
```
|
|
99
101
|
|
|
100
|
-
`latest` prints the current screenshot path
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
question. Open the printed image paths with the available image viewer.
|
|
102
|
+
`latest` prints the newest current screenshot's path. Use `list` to find a
|
|
103
|
+
specific comparison ID or browser variant. While changing UI, open the current
|
|
104
|
+
image first. Add `--image diff` or `--image all` when you need to compare it.
|
|
105
|
+
Open the printed paths with an image viewer.
|
|
105
106
|
|
|
106
|
-
`accept` replaces one local baseline and updates
|
|
107
|
-
when the task explicitly authorizes accepting a baseline
|
|
108
|
-
just to make a diff disappear:
|
|
107
|
+
`accept` replaces one local baseline and updates the report entry. Use it only
|
|
108
|
+
when the task explicitly authorizes accepting a baseline:
|
|
109
109
|
|
|
110
110
|
```bash
|
|
111
111
|
vizzly tdd screenshots accept <comparison-id>
|
|
112
112
|
```
|
|
113
113
|
|
|
114
|
-
|
|
114
|
+
Don't accept a change just to make a diff disappear.
|
|
115
|
+
|
|
116
|
+
When the task calls for a cloud build:
|
|
115
117
|
|
|
116
118
|
```bash
|
|
117
|
-
vizzly run "<existing visual test command>" --
|
|
119
|
+
vizzly run "<existing visual test command>" --json
|
|
118
120
|
vizzly api schema --json
|
|
119
121
|
```
|
|
120
122
|
|
|
121
|
-
|
|
123
|
+
To make visual differences fail CI, see the
|
|
124
|
+
[CI guide](ci.md#cloud-ci).
|
|
125
|
+
|
|
126
|
+
## Inspect a comparison
|
|
122
127
|
|
|
123
|
-
|
|
128
|
+
Find comparison and image requests in the schema:
|
|
124
129
|
|
|
125
130
|
```bash
|
|
126
131
|
vizzly api schema getComparisonContext --json
|
|
127
132
|
vizzly api schema sdk.getComparisonImage --json
|
|
128
133
|
```
|
|
129
134
|
|
|
130
|
-
|
|
135
|
+
Request `comparison.images` to check which current, baseline, and diff images
|
|
136
|
+
are available. Download them through `sdk.getComparisonImage`, using the
|
|
137
|
+
comparison ID and image kind. Include the version header from the schema and
|
|
138
|
+
use `--output <new-file>` to save each image.
|
|
131
139
|
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
- Diff: `comparison.analysis.diff_image_url`
|
|
140
|
+
View the available images together. A new capture may have no baseline, and an
|
|
141
|
+
identical capture may have no diff. Report missing or inaccessible images;
|
|
142
|
+
don't invent them. Request `comparison.review` to check the review decision.
|
|
136
143
|
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
Previous review decisions help explain recurring evidence but do not decide the
|
|
140
|
-
current review.
|
|
144
|
+
Changed regions and full analysis use separate requests. Discover them when
|
|
145
|
+
the images alone don't answer the question:
|
|
141
146
|
|
|
142
|
-
|
|
147
|
+
```bash
|
|
148
|
+
vizzly api schema sdk.getComparisonRegions --json
|
|
149
|
+
vizzly api schema sdk.getComparisonAnalysis --json
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Use `--output <new-file>` for full analysis. Compare those details with the
|
|
153
|
+
images and earlier comparisons. Previous decisions can help explain a recurring
|
|
154
|
+
change; they don't decide the current review.
|
|
155
|
+
|
|
156
|
+
Other useful commands:
|
|
143
157
|
|
|
144
158
|
```bash
|
|
145
159
|
vizzly context screenshot "<screenshot-name>" --source <local-or-cloud> --json
|
|
@@ -147,10 +161,6 @@ vizzly context similar <fingerprint-hash> --source cloud --json
|
|
|
147
161
|
vizzly context review-queue --source <local-or-cloud> --json
|
|
148
162
|
```
|
|
149
163
|
|
|
150
|
-
Use
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
## Continue Without Guessing
|
|
154
|
-
|
|
155
|
-
Use IDs and pagination values from API responses. Keep cursors opaque and
|
|
156
|
-
preserve the query they belong to when requesting the next page.
|
|
164
|
+
Use IDs and page values from the API response. Pass cursors back unchanged,
|
|
165
|
+
keep the same query, and request only what you need. Fetch comments when they
|
|
166
|
+
help explain a review.
|
|
@@ -1,28 +1,27 @@
|
|
|
1
|
-
# Dynamic
|
|
1
|
+
# Dynamic content
|
|
2
2
|
|
|
3
3
|
Dates, timers, random data, API content, generated images, and responsive text
|
|
4
|
-
can cause
|
|
4
|
+
can cause diffs. Check the images before deciding that's what happened.
|
|
5
5
|
|
|
6
|
-
##
|
|
6
|
+
## Check before changing settings
|
|
7
7
|
|
|
8
|
-
1.
|
|
8
|
+
1. Open the images and check screenshot history:
|
|
9
9
|
|
|
10
10
|
```bash
|
|
11
11
|
vizzly context screenshot "<screenshot-name>" --source <local-or-cloud> --json
|
|
12
12
|
```
|
|
13
13
|
|
|
14
|
-
2.
|
|
15
|
-
|
|
16
|
-
3.
|
|
17
|
-
4. Use diff regions
|
|
18
|
-
|
|
19
|
-
5.
|
|
20
|
-
|
|
14
|
+
2. Note where the change appears, how often it happens, the capture details,
|
|
15
|
+
and whether it appears in other builds or variants.
|
|
16
|
+
3. Use fixed test data when it can remove changes unrelated to the test.
|
|
17
|
+
4. Use diff regions to find changed pixels. They don't tell you why the pixels
|
|
18
|
+
changed or whether to approve them.
|
|
19
|
+
5. Keep `threshold` and `minClusterSize` unless repeated captures show they
|
|
20
|
+
need to change.
|
|
21
21
|
|
|
22
|
-
Avoid broad masks
|
|
23
|
-
|
|
24
|
-
|
|
22
|
+
Avoid broad masks or global threshold changes to hide a problem in one region.
|
|
23
|
+
Metadata alone can't explain a visual change. A recurring change can still be
|
|
24
|
+
a bug, especially if moving content changes nearby layout.
|
|
25
25
|
|
|
26
|
-
|
|
27
|
-
timing, or baseline
|
|
28
|
-
it.
|
|
26
|
+
Describe what you see first. Treat changing test data, layout shifts, capture
|
|
27
|
+
timing, or a wrong baseline as possible causes until you can confirm them.
|
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# SDK
|
|
1
|
+
# SDK capture
|
|
2
2
|
|
|
3
|
-
Use the
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
Use the repo's existing Vizzly integration and test flow. Use this JavaScript
|
|
4
|
+
example only if the repo doesn't already capture screenshots through a
|
|
5
|
+
framework integration.
|
|
6
6
|
|
|
7
7
|
## JavaScript
|
|
8
8
|
|
|
@@ -14,28 +14,28 @@ await vizzlyScreenshot('settings-profile-edit-mode', screenshot);
|
|
|
14
14
|
```
|
|
15
15
|
|
|
16
16
|
`vizzlyScreenshot(name, image, options)` accepts PNG bytes or a file path.
|
|
17
|
-
Options include `properties`, `threshold`, `minClusterSize`, and
|
|
18
|
-
|
|
19
|
-
it.
|
|
17
|
+
Options include `properties`, `threshold`, `minClusterSize`, and `fullPage`.
|
|
18
|
+
Keep existing settings unless the images show why they need to change.
|
|
20
19
|
|
|
21
|
-
## Keep
|
|
20
|
+
## Keep names and baseline matching consistent
|
|
22
21
|
|
|
23
|
-
Use
|
|
24
|
-
browser.
|
|
25
|
-
properties
|
|
22
|
+
Use descriptive names and keep them the same across runs. Baseline matching
|
|
23
|
+
always uses the name, viewport width, and browser. To also match on custom
|
|
24
|
+
properties, use `signatureProperties` in `vizzly.config.js` for local baselines.
|
|
25
|
+
Cloud baselines use the project's signature settings. Other properties are
|
|
26
|
+
metadata.
|
|
26
27
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
28
|
+
Theme, locale, or state won't create a separate baseline unless the matching
|
|
29
|
+
settings include them. Avoid generic names like `screenshot1` and names with
|
|
30
|
+
slashes.
|
|
30
31
|
|
|
31
|
-
## Follow
|
|
32
|
+
## Follow the existing setup
|
|
32
33
|
|
|
33
|
-
|
|
34
|
+
Keep the Vitest matcher if it's already configured:
|
|
34
35
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
36
|
+
```javascript
|
|
37
|
+
await expect(page).toMatchScreenshot('hero-section.png');
|
|
38
|
+
```
|
|
38
39
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
- Inspect the repository's integration setup before changing capture code.
|
|
40
|
+
Check the repo's setup before changing capture code. Use its Storybook,
|
|
41
|
+
static-site, Ember, or Swift integration if it has one.
|
|
@@ -1,44 +0,0 @@
|
|
|
1
|
-
# Setup And CI
|
|
2
|
-
|
|
3
|
-
Use this reference only when the task includes initialization, CI changes, or
|
|
4
|
-
configuration troubleshooting. These workflows mutate files or external build
|
|
5
|
-
state.
|
|
6
|
-
|
|
7
|
-
## Initialize
|
|
8
|
-
|
|
9
|
-
Initialize only with authorization:
|
|
10
|
-
|
|
11
|
-
```bash
|
|
12
|
-
vizzly init
|
|
13
|
-
```
|
|
14
|
-
|
|
15
|
-
Use [CLI context](cli-context.md) for local evidence and TDD lifecycle commands.
|
|
16
|
-
|
|
17
|
-
## Cloud CI
|
|
18
|
-
|
|
19
|
-
Use a project-scoped token from the repository's existing secret system. Never
|
|
20
|
-
print it or write it into the repository.
|
|
21
|
-
|
|
22
|
-
```bash
|
|
23
|
-
vizzly run "<existing visual test command>" --wait --json
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
For parallel CI, give every shard the same workflow-specific parallel ID. Run
|
|
27
|
-
shards without `--wait`, then finalize once after all shards finish:
|
|
28
|
-
|
|
29
|
-
```bash
|
|
30
|
-
vizzly run "<shard test command>" --parallel-id "<shared-ci-run-id>" --json
|
|
31
|
-
vizzly finalize "<shared-ci-run-id>" --json
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
## Troubleshoot
|
|
35
|
-
|
|
36
|
-
- `vizzly doctor`: local configuration
|
|
37
|
-
- `vizzly tdd status --json`: a local daemon started by the task
|
|
38
|
-
- `vizzly status <build-id> --json`: cloud lifecycle
|
|
39
|
-
- `vizzly api schema --json`: discover supported cloud review operations and
|
|
40
|
-
their request and response schemas.
|
|
41
|
-
|
|
42
|
-
If screenshots are absent, verify the existing integration, the test path that
|
|
43
|
-
should capture them, and the active session. If authentication is absent,
|
|
44
|
-
report it rather than logging in unless setup is in scope.
|