td-ai-tools 1.1.10 → 1.1.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/skills/README.md +2 -2
- package/skills/{scry → visual-regression}/SKILL.md +33 -32
- package/skills/visual-regression/agents/openai.yaml +4 -0
- package/skills/visual-regression/scripts/__pycache__/visual_regression.cpython-312.pyc +0 -0
- package/skills/{scry/scripts/scry.py → visual-regression/scripts/visual_regression.py} +16 -16
- package/skills/{scry/tests/test_scry.py → visual-regression/tests/test_visual_regression.py} +18 -18
- package/skills/scry/agents/openai.yaml +0 -4
package/package.json
CHANGED
package/skills/README.md
CHANGED
|
@@ -15,11 +15,11 @@
|
|
|
15
15
|
- `pull-request`: Generates well-structured GitHub pull request descriptions for Shopify theme development teams by analyzing…
|
|
16
16
|
- `pull-request-statamic`: Generates GitHub pull request descriptions for Statamic and Laravel development by analyzing git diffs and…
|
|
17
17
|
- `record-changes`: Update `docs/changes.md` by summarizing the current branch against the primary development branch.
|
|
18
|
-
- `
|
|
19
|
-
- `shopify-cli`: Shopify CLI workflows for theme development. Use when the user needs to run or explain Shopify theme commands,…
|
|
18
|
+
- `shopify-cli`: Shopify CLI workflows for theme development.
|
|
20
19
|
- `stylesheet-migration`: Migrate Shopify Liquid `{% stylesheet %}` blocks into theme CSS assets using bundled Python scripts.
|
|
21
20
|
- `td-js-vanilla-rules`: Theory Digital vanilla JavaScript standards for Shopify theme work.
|
|
22
21
|
- `td-review`: Run parallel code review agents on a PR (including TD theme compliance) and produce a synthesized findings…
|
|
22
|
+
- `visual-regression`: Single-site visual regression workflow for comparing a live URL against a preview/staging URL in Playwright…
|
|
23
23
|
|
|
24
24
|
## Skill Structure Convention
|
|
25
25
|
- `<skill-name>/SKILL.md`
|
|
@@ -1,13 +1,14 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
3
|
-
version:
|
|
2
|
+
name: visual-regression
|
|
3
|
+
version: 2.0.0
|
|
4
4
|
description: Single-site visual regression workflow for comparing a live URL against a preview/staging URL in Playwright UI mode. Use when the user provides or asks to compare two links, one live/baseline link and one preview link, wants fresh or reused live snapshots, or needs an ad hoc visual check that stores generated paths and baselines inside the skill folder rather than adding a normal CI site.
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
#
|
|
7
|
+
# Visual Regression
|
|
8
8
|
|
|
9
|
-
Use `
|
|
9
|
+
Use `visual-regression` for temporary visual regression checks between one live site and one preview site.
|
|
10
10
|
It keeps state in this skill folder so the workflow is separate from the repo's regular `sites/` configs.
|
|
11
|
+
Prefer over-analyzing with thorough crawls to tightly scoped reviews.
|
|
11
12
|
|
|
12
13
|
## Workflow
|
|
13
14
|
|
|
@@ -16,7 +17,7 @@ It keeps state in this skill folder so the workflow is separate from the repo's
|
|
|
16
17
|
3. Run the bundled script from the repository root:
|
|
17
18
|
|
|
18
19
|
```bash
|
|
19
|
-
python3 .agents/skills/
|
|
20
|
+
python3 .agents/skills/visual-regression/scripts/visual_regression.py <live-url> <preview-url> --refresh
|
|
20
21
|
```
|
|
21
22
|
|
|
22
23
|
Use `--reuse` instead of `--refresh` when the user wants to compare against existing baselines.
|
|
@@ -24,11 +25,11 @@ The script opens Playwright UI mode by default.
|
|
|
24
25
|
|
|
25
26
|
## What The Script Does
|
|
26
27
|
|
|
27
|
-
- Writes `.agents/skills/
|
|
28
|
-
- On first run, generates `.agents/skills/
|
|
29
|
-
- Reuses `.agents/skills/
|
|
30
|
-
- Stores live baseline screenshots in `.agents/skills/
|
|
31
|
-
- Generates runtime Playwright files under `.agents/skills/
|
|
28
|
+
- Writes `.agents/skills/visual-regression/config.json` with the latest live and preview URLs.
|
|
29
|
+
- On first run, generates `.agents/skills/visual-regression/paths.json` from the live URL's sitemap.
|
|
30
|
+
- Reuses `.agents/skills/visual-regression/paths.json` on later runs unless `--regenerate-paths` is passed.
|
|
31
|
+
- Stores live baseline screenshots in `.agents/skills/visual-regression/baselines/`.
|
|
32
|
+
- Generates runtime Playwright files under `.agents/skills/visual-regression/runtime/`.
|
|
32
33
|
- Runs `npx playwright test --ui` against the generated runtime config.
|
|
33
34
|
|
|
34
35
|
If sitemap generation fails on the first run, the script falls back to a homepage-only `paths.json`.
|
|
@@ -39,25 +40,25 @@ After generation, inspect `paths.json` and trim duplicate template paths when th
|
|
|
39
40
|
Refresh baselines and open UI comparison:
|
|
40
41
|
|
|
41
42
|
```bash
|
|
42
|
-
python3 .agents/skills/
|
|
43
|
+
python3 .agents/skills/visual-regression/scripts/visual_regression.py https://example.com https://preview.example.com --refresh
|
|
43
44
|
```
|
|
44
45
|
|
|
45
46
|
Reuse existing baselines:
|
|
46
47
|
|
|
47
48
|
```bash
|
|
48
|
-
python3 .agents/skills/
|
|
49
|
+
python3 .agents/skills/visual-regression/scripts/visual_regression.py https://example.com https://preview.example.com --reuse
|
|
49
50
|
```
|
|
50
51
|
|
|
51
52
|
Regenerate paths from the live sitemap:
|
|
52
53
|
|
|
53
54
|
```bash
|
|
54
|
-
python3 .agents/skills/
|
|
55
|
+
python3 .agents/skills/visual-regression/scripts/visual_regression.py https://example.com https://preview.example.com --refresh --regenerate-paths
|
|
55
56
|
```
|
|
56
57
|
|
|
57
58
|
Run headless instead of UI mode:
|
|
58
59
|
|
|
59
60
|
```bash
|
|
60
|
-
python3 .agents/skills/
|
|
61
|
+
python3 .agents/skills/visual-regression/scripts/visual_regression.py https://example.com https://preview.example.com --refresh --no-ui
|
|
61
62
|
```
|
|
62
63
|
|
|
63
64
|
Useful options:
|
|
@@ -75,25 +76,25 @@ Some theme repositories do not have a local `package.json`, `node_modules/`, or
|
|
|
75
76
|
Create the temporary install directory:
|
|
76
77
|
|
|
77
78
|
```bash
|
|
78
|
-
mkdir -p /tmp/
|
|
79
|
+
mkdir -p /tmp/visual-regression-playwright
|
|
79
80
|
```
|
|
80
81
|
|
|
81
82
|
Install Playwright Test there:
|
|
82
83
|
|
|
83
84
|
```bash
|
|
84
|
-
npm install --prefix /tmp/
|
|
85
|
+
npm install --prefix /tmp/visual-regression-playwright @playwright/test
|
|
85
86
|
```
|
|
86
87
|
|
|
87
88
|
Install the browser needed for the run:
|
|
88
89
|
|
|
89
90
|
```bash
|
|
90
|
-
/tmp/
|
|
91
|
+
/tmp/visual-regression-playwright/node_modules/.bin/playwright install chromium
|
|
91
92
|
```
|
|
92
93
|
|
|
93
|
-
Generate the
|
|
94
|
+
Generate the visual regression config and paths from the repository root:
|
|
94
95
|
|
|
95
96
|
```bash
|
|
96
|
-
python3 .agents/skills/
|
|
97
|
+
python3 .agents/skills/visual-regression/scripts/visual_regression.py "$LIVE_URL" "$PREVIEW_URL" --refresh --regenerate-paths --no-ui --browser=chromium
|
|
97
98
|
```
|
|
98
99
|
|
|
99
100
|
If `paths.json` is large, inspect it and trim duplicate template paths before the full comparison.
|
|
@@ -101,13 +102,13 @@ If `paths.json` is large, inspect it and trim duplicate template paths before th
|
|
|
101
102
|
Run the generated suite with the temporary Playwright install:
|
|
102
103
|
|
|
103
104
|
```bash
|
|
104
|
-
env NODE_PATH=/tmp/
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
/tmp/
|
|
110
|
-
--config "$PWD/.agents/skills/
|
|
105
|
+
env NODE_PATH=/tmp/visual-regression-playwright/node_modules \
|
|
106
|
+
VISUAL_REGRESSION_CONFIG="$PWD/.agents/skills/visual-regression/config.json" \
|
|
107
|
+
VISUAL_REGRESSION_PATHS="$PWD/.agents/skills/visual-regression/paths.json" \
|
|
108
|
+
VISUAL_REGRESSION_BASELINE_DIR="$PWD/.agents/skills/visual-regression/baselines" \
|
|
109
|
+
VISUAL_REGRESSION_REFRESH=1 \
|
|
110
|
+
/tmp/visual-regression-playwright/node_modules/.bin/playwright test \
|
|
111
|
+
--config "$PWD/.agents/skills/visual-regression/runtime/playwright.config.cjs"
|
|
111
112
|
```
|
|
112
113
|
|
|
113
114
|
## Cleanup
|
|
@@ -121,7 +122,7 @@ rm -rf playwright-report test-results
|
|
|
121
122
|
Remove the temporary dependency install:
|
|
122
123
|
|
|
123
124
|
```bash
|
|
124
|
-
rm -rf /tmp/
|
|
125
|
+
rm -rf /tmp/visual-regression-playwright
|
|
125
126
|
```
|
|
126
127
|
|
|
127
128
|
Only remove downloaded Playwright browsers when intentionally resetting local tooling:
|
|
@@ -130,12 +131,12 @@ Only remove downloaded Playwright browsers when intentionally resetting local to
|
|
|
130
131
|
rm -rf ~/.cache/ms-playwright
|
|
131
132
|
```
|
|
132
133
|
|
|
133
|
-
Keep these
|
|
134
|
+
Keep these visual regression artifacts unless intentionally resetting the comparison state:
|
|
134
135
|
|
|
135
|
-
- `.agents/skills/
|
|
136
|
-
- `.agents/skills/
|
|
137
|
-
- `.agents/skills/
|
|
138
|
-
- `.agents/skills/
|
|
136
|
+
- `.agents/skills/visual-regression/config.json`
|
|
137
|
+
- `.agents/skills/visual-regression/paths.json`
|
|
138
|
+
- `.agents/skills/visual-regression/runtime/`
|
|
139
|
+
- `.agents/skills/visual-regression/baselines/`
|
|
139
140
|
|
|
140
141
|
## Playwright CLI
|
|
141
142
|
|
|
Binary file
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
|
-
"""Ad hoc visual regression runner for the
|
|
2
|
+
"""Ad hoc visual regression runner for the visual-regression skill.
|
|
3
3
|
|
|
4
4
|
The Python entrypoint owns URL/path discovery and writes the runtime Playwright
|
|
5
5
|
files used for comparison. Playwright still runs through `npx playwright test`
|
|
@@ -39,9 +39,9 @@ PLAYWRIGHT_SPEC = r'''const { test, expect } = require("@playwright/test");
|
|
|
39
39
|
const fs = require("fs");
|
|
40
40
|
const path = require("path");
|
|
41
41
|
|
|
42
|
-
const config = JSON.parse(fs.readFileSync(process.env.
|
|
43
|
-
const paths = JSON.parse(fs.readFileSync(process.env.
|
|
44
|
-
const refresh = process.env.
|
|
42
|
+
const config = JSON.parse(fs.readFileSync(process.env.VISUAL_REGRESSION_CONFIG, "utf8"));
|
|
43
|
+
const paths = JSON.parse(fs.readFileSync(process.env.VISUAL_REGRESSION_PATHS, "utf8"));
|
|
44
|
+
const refresh = process.env.VISUAL_REGRESSION_REFRESH === "1";
|
|
45
45
|
|
|
46
46
|
function joinUrl(baseUrl, pagePath) {
|
|
47
47
|
const base = new URL(baseUrl);
|
|
@@ -179,7 +179,7 @@ for (const pathEntry of paths) {
|
|
|
179
179
|
|
|
180
180
|
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
|
|
181
181
|
parser = argparse.ArgumentParser(
|
|
182
|
-
prog="
|
|
182
|
+
prog="visual_regression.py",
|
|
183
183
|
formatter_class=argparse.RawDescriptionHelpFormatter,
|
|
184
184
|
description=(
|
|
185
185
|
"Compare a live URL against a preview URL with generated "
|
|
@@ -292,7 +292,7 @@ def fetch_xml(url: str) -> str:
|
|
|
292
292
|
request = urllib.request.Request(
|
|
293
293
|
url,
|
|
294
294
|
headers={
|
|
295
|
-
"User-Agent": "
|
|
295
|
+
"User-Agent": "VisualRegression/1.0",
|
|
296
296
|
"Accept": "application/xml, text/xml, */*",
|
|
297
297
|
},
|
|
298
298
|
)
|
|
@@ -417,7 +417,7 @@ def resolve_refresh_choice(args: argparse.Namespace) -> bool:
|
|
|
417
417
|
if isinstance(args.refresh, bool):
|
|
418
418
|
return args.refresh
|
|
419
419
|
if not sys.stdin.isatty():
|
|
420
|
-
raise RuntimeError("Choose --refresh or --reuse. In non-interactive runs,
|
|
420
|
+
raise RuntimeError("Choose --refresh or --reuse. In non-interactive runs, visual regression will not guess.")
|
|
421
421
|
answer = input("Refresh live baselines before comparing? [Y/n] ")
|
|
422
422
|
return not re.match(r"^n(o)?$", answer.strip(), re.I)
|
|
423
423
|
|
|
@@ -433,7 +433,7 @@ def browser_project(browser: str) -> str:
|
|
|
433
433
|
def write_runtime_files(config: dict[str, Any]) -> Path:
|
|
434
434
|
RUNTIME_DIR.mkdir(parents=True, exist_ok=True)
|
|
435
435
|
|
|
436
|
-
spec_path = RUNTIME_DIR / "
|
|
436
|
+
spec_path = RUNTIME_DIR / "visual-regression.spec.cjs"
|
|
437
437
|
playwright_config_path = RUNTIME_DIR / "playwright.config.cjs"
|
|
438
438
|
|
|
439
439
|
spec_path.write_text(PLAYWRIGHT_SPEC, encoding="utf-8")
|
|
@@ -472,16 +472,16 @@ def run_playwright(playwright_config_path: Path, ui: bool, refresh: bool) -> int
|
|
|
472
472
|
env = os.environ.copy()
|
|
473
473
|
env.update(
|
|
474
474
|
{
|
|
475
|
-
"
|
|
476
|
-
"
|
|
477
|
-
"
|
|
478
|
-
"
|
|
475
|
+
"VISUAL_REGRESSION_CONFIG": str(CONFIG_FILE),
|
|
476
|
+
"VISUAL_REGRESSION_PATHS": str(PATHS_FILE),
|
|
477
|
+
"VISUAL_REGRESSION_BASELINE_DIR": str(BASELINE_DIR),
|
|
478
|
+
"VISUAL_REGRESSION_REFRESH": "1" if refresh else "0",
|
|
479
479
|
}
|
|
480
480
|
)
|
|
481
481
|
try:
|
|
482
482
|
completed = subprocess.run(args, env=env)
|
|
483
483
|
except FileNotFoundError as exc:
|
|
484
|
-
raise RuntimeError("npx was not found; install Node.js/npm before running
|
|
484
|
+
raise RuntimeError("npx was not found; install Node.js/npm before running visual regression.") from exc
|
|
485
485
|
return completed.returncode
|
|
486
486
|
|
|
487
487
|
|
|
@@ -522,8 +522,8 @@ def main(argv: list[str] | None = None) -> int:
|
|
|
522
522
|
write_json(CONFIG_FILE, config)
|
|
523
523
|
playwright_config_path = write_runtime_files(config)
|
|
524
524
|
|
|
525
|
-
print(f"
|
|
526
|
-
print(f"
|
|
525
|
+
print(f"Visual regression paths: {PATHS_FILE} ({len(paths)} paths)")
|
|
526
|
+
print(f"Visual regression baselines: {BASELINE_DIR}")
|
|
527
527
|
print(
|
|
528
528
|
"Refreshing live baselines during this run."
|
|
529
529
|
if refresh
|
|
@@ -539,5 +539,5 @@ if __name__ == "__main__":
|
|
|
539
539
|
except KeyboardInterrupt:
|
|
540
540
|
sys.exit(130)
|
|
541
541
|
except Exception as exc:
|
|
542
|
-
print(f"
|
|
542
|
+
print(f"visual-regression: {exc}", file=sys.stderr)
|
|
543
543
|
sys.exit(1)
|
package/skills/{scry/tests/test_scry.py → visual-regression/tests/test_visual_regression.py}
RENAMED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
"""Tests for
|
|
1
|
+
"""Tests for visual_regression.py helper behavior.
|
|
2
2
|
|
|
3
|
-
Run: python3 skills/
|
|
3
|
+
Run: python3 skills/visual-regression/tests/test_visual_regression.py
|
|
4
4
|
"""
|
|
5
5
|
|
|
6
6
|
from __future__ import annotations
|
|
@@ -10,20 +10,20 @@ import sys
|
|
|
10
10
|
from pathlib import Path
|
|
11
11
|
|
|
12
12
|
|
|
13
|
-
_SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "
|
|
14
|
-
_spec = importlib.util.spec_from_file_location("
|
|
13
|
+
_SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "visual_regression.py"
|
|
14
|
+
_spec = importlib.util.spec_from_file_location("visual_regression", _SCRIPT)
|
|
15
15
|
assert _spec and _spec.loader
|
|
16
|
-
|
|
17
|
-
_spec.loader.exec_module(
|
|
16
|
+
visual_regression = importlib.util.module_from_spec(_spec)
|
|
17
|
+
_spec.loader.exec_module(visual_regression)
|
|
18
18
|
|
|
19
19
|
|
|
20
20
|
def test_normalize_url_adds_origin_slash():
|
|
21
|
-
assert
|
|
21
|
+
assert visual_regression.normalize_url("https://example.com", "live URL") == "https://example.com/"
|
|
22
22
|
|
|
23
23
|
|
|
24
24
|
def test_normalize_url_rejects_non_http():
|
|
25
25
|
try:
|
|
26
|
-
|
|
26
|
+
visual_regression.normalize_url("ftp://example.com", "live URL")
|
|
27
27
|
except ValueError as exc:
|
|
28
28
|
assert "http or https" in str(exc)
|
|
29
29
|
else:
|
|
@@ -31,14 +31,14 @@ def test_normalize_url_rejects_non_http():
|
|
|
31
31
|
|
|
32
32
|
|
|
33
33
|
def test_normalize_url_path_and_name():
|
|
34
|
-
assert
|
|
35
|
-
assert
|
|
36
|
-
assert
|
|
37
|
-
assert
|
|
34
|
+
assert visual_regression.normalize_url_path("about/team/") == "/about/team"
|
|
35
|
+
assert visual_regression.normalize_url_path("https://example.com/products/widget/") == "/products/widget"
|
|
36
|
+
assert visual_regression.path_name("/") == "homepage"
|
|
37
|
+
assert visual_regression.path_name("/products/widget!") == "products-widget"
|
|
38
38
|
|
|
39
39
|
|
|
40
40
|
def test_unique_sorted_paths_dedupes_by_path():
|
|
41
|
-
paths =
|
|
41
|
+
paths = visual_regression.unique_sorted_paths(
|
|
42
42
|
[
|
|
43
43
|
{"name": "z", "path": "/about/"},
|
|
44
44
|
{"name": "a", "path": "/"},
|
|
@@ -63,19 +63,19 @@ def test_parse_sitemap_index_recurses_and_filters_origin():
|
|
|
63
63
|
<url><loc>https://external.example.com/ignore/</loc></url>
|
|
64
64
|
</urlset>""",
|
|
65
65
|
}
|
|
66
|
-
original_fetch_xml =
|
|
66
|
+
original_fetch_xml = visual_regression.fetch_xml
|
|
67
67
|
try:
|
|
68
|
-
|
|
69
|
-
assert
|
|
68
|
+
visual_regression.fetch_xml = lambda url: fixtures[url]
|
|
69
|
+
assert visual_regression.parse_sitemap(
|
|
70
70
|
"https://example.com/sitemap.xml",
|
|
71
71
|
"https://example.com/",
|
|
72
72
|
) == [{"name": "about", "path": "/about"}]
|
|
73
73
|
finally:
|
|
74
|
-
|
|
74
|
+
visual_regression.fetch_xml = original_fetch_xml
|
|
75
75
|
|
|
76
76
|
|
|
77
77
|
def test_parse_args_defaults_browsers():
|
|
78
|
-
args =
|
|
78
|
+
args = visual_regression.parse_args(["https://a.test", "https://b.test", "--reuse"])
|
|
79
79
|
assert args.refresh is False
|
|
80
80
|
assert args.browsers == ["chromium", "firefox", "webkit"]
|
|
81
81
|
|