td-ai-tools 1.1.7 → 1.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/skills/README.md +2 -0
- package/skills/barrage/SKILL.md +180 -0
- package/skills/barrage/agents/openai.yaml +4 -0
- package/skills/barrage/scripts/__pycache__/build_queue.cpython-312.pyc +0 -0
- package/skills/barrage/scripts/build_queue.py +276 -0
- package/skills/barrage/tests/test_build_queue.py +135 -0
- package/skills/scry/SKILL.md +74 -0
- package/skills/scry/agents/openai.yaml +4 -0
- package/skills/scry/scripts/scry.py +542 -0
- package/skills/scry/tests/test_scry.py +96 -0
package/package.json
CHANGED
package/skills/README.md
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# skills
|
|
2
2
|
|
|
3
3
|
## Available Skills
|
|
4
|
+
- `barrage`: Convert a group of Basecamp todos or cards into a td-barrage queue.json task file.
|
|
4
5
|
- `basecamp`: Interact with Basecamp via the Basecamp CLI.
|
|
5
6
|
- `cache-reset`: Clear and warm Laravel and Statamic caches (including Statamic Glide image caches) after content or template…
|
|
6
7
|
- `car-ticket-generator`: Generate a ticket for the codex-auto-runner queue
|
|
@@ -12,6 +13,7 @@
|
|
|
12
13
|
- `pull-request`: Generates well-structured GitHub pull request descriptions for Shopify theme development teams by analyzing…
|
|
13
14
|
- `pull-request-statamic`: Generates GitHub pull request descriptions for Statamic and Laravel development by analyzing git diffs and…
|
|
14
15
|
- `record-changes`: Update `docs/changes.md` by summarizing the current branch against the primary development branch.
|
|
16
|
+
- `scry`: Single-site visual regression workflow for comparing a live URL against a preview/staging URL in Playwright…
|
|
15
17
|
- `stylesheet-migration`: Migrate Shopify Liquid `{% stylesheet %}` blocks into theme CSS assets using bundled Python scripts.
|
|
16
18
|
- `td-js-vanilla-rules`: Theory Digital vanilla JavaScript standards for Shopify theme work.
|
|
17
19
|
- `td-review`: Run parallel code review agents on a PR (including TD theme compliance) and produce a synthesized findings…
|
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: barrage
|
|
3
|
+
description: |
|
|
4
|
+
Convert a group of Basecamp todos or cards into a td-barrage queue.json task
|
|
5
|
+
file. Fetches the items via the Basecamp CLI, maps each title -> task id and
|
|
6
|
+
description -> prompt, and writes queue.json into this skill's directory. Use
|
|
7
|
+
when the user wants to turn Basecamp work into a barrage queue, build a
|
|
8
|
+
td-barrage queue from Basecamp, or run a todolist/card table through td-barrage.
|
|
9
|
+
invocable: true
|
|
10
|
+
argument-hint: "[basecamp todolist/card-table/column URL] [--timeout-mins N]"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# /barrage — Basecamp → td-barrage queue builder
|
|
14
|
+
|
|
15
|
+
Turns a user-specified group of Basecamp todos or cards into a `queue.json` in the
|
|
16
|
+
format consumed by **td-barrage**, a batch task runner (see
|
|
17
|
+
[Output: the td-barrage queue.json format](#output-the-td-barrage-queuejson-format)
|
|
18
|
+
below for the full schema — this skill emits that format directly and needs no
|
|
19
|
+
other repo installed).
|
|
20
|
+
|
|
21
|
+
## Input: the expected Basecamp item shape
|
|
22
|
+
|
|
23
|
+
This skill is designed for Basecamp items whose **titles carry a trailing `[Nh]`
|
|
24
|
+
estimate suffix** — e.g. `Build the checkout page [2h]` or `Fix nav [1.5h]`. That
|
|
25
|
+
convention comes from an estimation workflow (the **td-augury** generator produces
|
|
26
|
+
titles in exactly this shape), but nothing here requires that tool: any Basecamp
|
|
27
|
+
todo or card works. Titles *without* a suffix are used verbatim; titles *with* one
|
|
28
|
+
have it stripped from the task id by default (see `--keep-estimate`).
|
|
29
|
+
|
|
30
|
+
Concretely, each item is read from the Basecamp CLI's JSON output, which exposes:
|
|
31
|
+
|
|
32
|
+
- `title` — the display title (todos and cards)
|
|
33
|
+
- `content` — for todos this equals the title; for cards it holds the card body (HTML)
|
|
34
|
+
- `description` — the long-form body when present (preferred for the prompt)
|
|
35
|
+
|
|
36
|
+
## What it produces
|
|
37
|
+
|
|
38
|
+
Each Basecamp item becomes one td-barrage task:
|
|
39
|
+
|
|
40
|
+
| td-barrage field | Source | Notes |
|
|
41
|
+
|------------------|--------|-------|
|
|
42
|
+
| `id` | item **title** | trailing `[Nh]` estimate suffix stripped; deduped if needed |
|
|
43
|
+
| `prompt` | item **description** | HTML flattened to plain text |
|
|
44
|
+
| `success` | constant | always `{ "strategy": "default" }` |
|
|
45
|
+
| `timeoutMs` | constant | `35` minutes → `2100000` ms, unless the user specifies otherwise |
|
|
46
|
+
| `maxAttempts` | constant | `2` |
|
|
47
|
+
|
|
48
|
+
> The user may refer to these as `barrage_id`, `success_criteria`, `timeout`, and
|
|
49
|
+
> `max_attempts`. The actual td-barrage schema uses `id`, `success.strategy`,
|
|
50
|
+
> `timeoutMs` (**milliseconds**), and `maxAttempts` — this skill emits the real
|
|
51
|
+
> field names (documented in full below).
|
|
52
|
+
|
|
53
|
+
The file is written to **this skill's directory** as `queue.json`
|
|
54
|
+
(`build_queue.py` resolves that path from its own location, so it lands next to the
|
|
55
|
+
script regardless of which skills copy is invoked).
|
|
56
|
+
|
|
57
|
+
## Workflow
|
|
58
|
+
|
|
59
|
+
### 1. Determine the group and item type
|
|
60
|
+
|
|
61
|
+
Ask the user (if not already clear) for the Basecamp group to convert and whether
|
|
62
|
+
it is **todos** or **cards**. A "group" is normally one of:
|
|
63
|
+
|
|
64
|
+
- a **todolist** URL → its todos
|
|
65
|
+
- a **card table** URL → all its cards
|
|
66
|
+
- a **column** URL → the cards in that column
|
|
67
|
+
- a whole **project** → all todos or all cards in it
|
|
68
|
+
|
|
69
|
+
### 2. Resolve IDs from the URL
|
|
70
|
+
|
|
71
|
+
Always parse a URL first to extract the project and recording IDs:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
basecamp url parse "<url>" --json
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
This returns `project_id`, `type`, and `recording_id`.
|
|
78
|
+
|
|
79
|
+
### 3. Fetch the items as JSON
|
|
80
|
+
|
|
81
|
+
Use the matching list command. **Always pass `--json`.**
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
# Todos in a specific todolist
|
|
85
|
+
basecamp todos list --list <todolist_id> --in <project_id> --json > items.json
|
|
86
|
+
|
|
87
|
+
# All todos in a project
|
|
88
|
+
basecamp todos list --in <project_id> --json > items.json
|
|
89
|
+
|
|
90
|
+
# Cards in a specific card table (required flag if a project has >1 table)
|
|
91
|
+
basecamp cards list --card-table <table_id> --in <project_id> --json > items.json
|
|
92
|
+
|
|
93
|
+
# Cards in a single column
|
|
94
|
+
basecamp cards list --column <column_id> --in <project_id> --json > items.json
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
The CLI returns an envelope `{ ok, data, ... }`; the converter reads `data`
|
|
98
|
+
automatically (it also accepts a bare array or `--agent` raw output). By default
|
|
99
|
+
all (non-completed) items are returned — add `--all` for large lists, or filter
|
|
100
|
+
(e.g. `--status`) if the user only wants some.
|
|
101
|
+
|
|
102
|
+
### 4. Convert to queue.json
|
|
103
|
+
|
|
104
|
+
Pipe the fetched JSON into the converter (or pass `--input`):
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
python3 "$SKILL_DIR/scripts/build_queue.py" --input items.json
|
|
108
|
+
# or
|
|
109
|
+
basecamp todos list --list <id> --in <proj> --json \
|
|
110
|
+
| python3 "$SKILL_DIR/scripts/build_queue.py"
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
`$SKILL_DIR` is the directory containing this `SKILL.md`. The script writes
|
|
114
|
+
`queue.json` into that directory and prints a summary plus any per-item warnings
|
|
115
|
+
(skipped items with no description, deduped ids, etc.).
|
|
116
|
+
|
|
117
|
+
Options:
|
|
118
|
+
|
|
119
|
+
- `--timeout-mins N` — override the 35-minute default (the script converts to ms).
|
|
120
|
+
- `--max-attempts N` — override the default of 2.
|
|
121
|
+
- `--keep-estimate` — keep the `[Nh]` suffix in the id instead of stripping it.
|
|
122
|
+
- `--output PATH` — write somewhere other than `<skill>/queue.json`.
|
|
123
|
+
|
|
124
|
+
### 5. Report
|
|
125
|
+
|
|
126
|
+
Tell the user where `queue.json` was written, how many tasks it contains, and
|
|
127
|
+
surface any warnings the script printed (skipped or renamed items).
|
|
128
|
+
|
|
129
|
+
## Output: the td-barrage queue.json format
|
|
130
|
+
|
|
131
|
+
td-barrage validates the file with a Zod schema. This skill emits the minimal
|
|
132
|
+
valid subset; the runner fills in the rest with the defaults shown below. The full
|
|
133
|
+
task schema is:
|
|
134
|
+
|
|
135
|
+
```jsonc
|
|
136
|
+
{
|
|
137
|
+
"tasks": [
|
|
138
|
+
{
|
|
139
|
+
"id": "string", // required, non-empty; must be unique across the file
|
|
140
|
+
"prompt": "string", // required, non-empty
|
|
141
|
+
"dependsOn": ["string"], // optional, default [] (ids of prerequisite tasks)
|
|
142
|
+
"maxAttempts": 2, // optional, integer >= 1
|
|
143
|
+
"timeoutMs": 2100000, // optional, positive integer (MILLISECONDS)
|
|
144
|
+
"success": { // optional, default { "strategy": "default" }
|
|
145
|
+
"strategy": "default" // one of: "default" | "file_exists" | "result_json"
|
|
146
|
+
// "file_exists" also needs: "path": "string"
|
|
147
|
+
// "result_json" also needs: "path": "string"
|
|
148
|
+
},
|
|
149
|
+
"cwd": "string", // optional, working directory for the task
|
|
150
|
+
"resultPath": "string" // optional, where the task writes its result
|
|
151
|
+
}
|
|
152
|
+
]
|
|
153
|
+
}
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
The top-level value may be either `{ "tasks": [...] }` (what this skill writes) or a
|
|
157
|
+
bare `[...]` array of tasks. Runtime-managed fields (`status`, `attempts`, `error`,
|
|
158
|
+
`startedAt`, `finishedAt`, `recoveryNotes`) are set by td-barrage and should not be
|
|
159
|
+
authored here.
|
|
160
|
+
|
|
161
|
+
Validation rules enforced by the runner:
|
|
162
|
+
|
|
163
|
+
- **Unique ids** — duplicate `id` values are rejected (`Duplicate task id`). The
|
|
164
|
+
converter pre-empts this by renaming collisions to `"<title> (2)"`.
|
|
165
|
+
- **No self-dependency** — a task cannot list its own id in `dependsOn`.
|
|
166
|
+
- **No dangling dependencies** — every id in `dependsOn` must match another task's id.
|
|
167
|
+
|
|
168
|
+
## Notes & caveats
|
|
169
|
+
|
|
170
|
+
- **Empty descriptions are skipped.** td-barrage requires a non-empty `prompt`, so
|
|
171
|
+
items with no description cannot become tasks. The script warns for each one —
|
|
172
|
+
relay this so the user can add descriptions in Basecamp and re-run if needed.
|
|
173
|
+
- **Duplicate titles** would make td-barrage reject the whole file
|
|
174
|
+
(`Duplicate task id`). The script renames collisions to `"<title> (2)"` and
|
|
175
|
+
warns; prefer fixing the source titles when that happens.
|
|
176
|
+
- **No dependencies are emitted.** Every task is independent (`dependsOn` defaults
|
|
177
|
+
to `[]`). If the user wants an execution order, edit `queue.json` afterward to add
|
|
178
|
+
`dependsOn` arrays referencing other task ids.
|
|
179
|
+
- The Basecamp CLI is already authenticated; verify with
|
|
180
|
+
`basecamp auth status --json` if a fetch fails.
|
|
Binary file
|
|
@@ -0,0 +1,276 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Convert Basecamp todos/cards into a td-barrage queue.json.
|
|
3
|
+
|
|
4
|
+
Reads the JSON emitted by the Basecamp CLI (`basecamp todos list ... --json`,
|
|
5
|
+
`basecamp cards list ... --json`, or their `--agent` raw form) and writes a
|
|
6
|
+
td-barrage task file. The only runtime dependency is the Basecamp CLI that
|
|
7
|
+
produced the input; the queue.json format it writes is documented in ../SKILL.md
|
|
8
|
+
("Output: the td-barrage queue.json format") so no other repo need be installed.
|
|
9
|
+
|
|
10
|
+
Field mapping (see ../SKILL.md):
|
|
11
|
+
- id <- the item title, with any trailing `[Nh]` estimate suffix stripped
|
|
12
|
+
- prompt <- the item description (HTML stripped to plain text)
|
|
13
|
+
- success <- {"strategy": "default"} on every item
|
|
14
|
+
- timeoutMs <- --timeout-mins * 60 * 1000 (default 35 minutes)
|
|
15
|
+
- maxAttempts <- --max-attempts (default 2)
|
|
16
|
+
|
|
17
|
+
The Basecamp CLI normalizes both todos and cards to expose `title`, `content`,
|
|
18
|
+
and `description`. For todos `content == title`; for cards the body lives in
|
|
19
|
+
`content` (HTML). We therefore prefer `description` for the prompt and only fall
|
|
20
|
+
back to `content` when it differs from the title.
|
|
21
|
+
"""
|
|
22
|
+
|
|
23
|
+
from __future__ import annotations
|
|
24
|
+
|
|
25
|
+
import argparse
|
|
26
|
+
import json
|
|
27
|
+
import re
|
|
28
|
+
import sys
|
|
29
|
+
from html.parser import HTMLParser
|
|
30
|
+
from pathlib import Path
|
|
31
|
+
from typing import Any
|
|
32
|
+
|
|
33
|
+
# Skill root is the parent of this script's directory (scripts/ -> <skill>/).
|
|
34
|
+
SKILL_DIR = Path(__file__).resolve().parent.parent
|
|
35
|
+
|
|
36
|
+
# Matches a trailing estimate suffix like " [2h]" or " [1.5h]" (case-insensitive).
|
|
37
|
+
# Basecamp titles produced by an estimation workflow carry this suffix (the
|
|
38
|
+
# td-augury generator appends "[<hours>h]"); we strip it from the task id so the
|
|
39
|
+
# id reads as a plain title. Titles without a suffix are unaffected.
|
|
40
|
+
ESTIMATE_SUFFIX_RE = re.compile(r"\s*\[\d+(?:\.\d+)?h\]\s*$", re.IGNORECASE)
|
|
41
|
+
|
|
42
|
+
# Block-level tags whose close (or self-close) should become a line break when
|
|
43
|
+
# flattening an HTML card body to plain text.
|
|
44
|
+
_BLOCK_TAGS = {
|
|
45
|
+
"p", "div", "br", "li", "ul", "ol", "tr", "h1", "h2", "h3", "h4", "h5", "h6",
|
|
46
|
+
"blockquote", "pre", "section", "article", "header", "footer",
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
class _TextExtractor(HTMLParser):
|
|
51
|
+
"""Collapse an HTML fragment to readable plain text, preserving line breaks."""
|
|
52
|
+
|
|
53
|
+
def __init__(self) -> None:
|
|
54
|
+
super().__init__(convert_charrefs=True)
|
|
55
|
+
self._parts: list[str] = []
|
|
56
|
+
|
|
57
|
+
def handle_data(self, data: str) -> None:
|
|
58
|
+
self._parts.append(data)
|
|
59
|
+
|
|
60
|
+
def handle_starttag(self, tag: str, attrs: Any) -> None:
|
|
61
|
+
if tag == "br":
|
|
62
|
+
self._parts.append("\n")
|
|
63
|
+
|
|
64
|
+
def handle_endtag(self, tag: str) -> None:
|
|
65
|
+
if tag in _BLOCK_TAGS:
|
|
66
|
+
self._parts.append("\n")
|
|
67
|
+
|
|
68
|
+
def text(self) -> str:
|
|
69
|
+
return "".join(self._parts)
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
def html_to_text(value: str) -> str:
|
|
73
|
+
"""Strip HTML tags/entities and normalize whitespace.
|
|
74
|
+
|
|
75
|
+
Plain-text input passes through unchanged apart from whitespace cleanup, so
|
|
76
|
+
it is safe to run on both todo descriptions (plain) and card bodies (HTML).
|
|
77
|
+
"""
|
|
78
|
+
parser = _TextExtractor()
|
|
79
|
+
parser.feed(value)
|
|
80
|
+
text = parser.text()
|
|
81
|
+
# Collapse runs of blank lines and trim trailing spaces on each line.
|
|
82
|
+
lines = [line.strip() for line in text.splitlines()]
|
|
83
|
+
out: list[str] = []
|
|
84
|
+
blank = False
|
|
85
|
+
for line in lines:
|
|
86
|
+
if line:
|
|
87
|
+
out.append(line)
|
|
88
|
+
blank = False
|
|
89
|
+
elif not blank and out:
|
|
90
|
+
out.append("")
|
|
91
|
+
blank = True
|
|
92
|
+
return "\n".join(out).strip()
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
def first_nonempty(record: dict[str, Any], keys: tuple[str, ...]) -> str:
|
|
96
|
+
for key in keys:
|
|
97
|
+
value = record.get(key)
|
|
98
|
+
if isinstance(value, str) and value.strip():
|
|
99
|
+
return value.strip()
|
|
100
|
+
return ""
|
|
101
|
+
|
|
102
|
+
|
|
103
|
+
def extract_title(record: dict[str, Any]) -> str:
|
|
104
|
+
"""The display title (before estimate-suffix stripping)."""
|
|
105
|
+
return first_nonempty(record, ("title", "content", "name"))
|
|
106
|
+
|
|
107
|
+
|
|
108
|
+
def extract_description(record: dict[str, Any], title: str) -> str:
|
|
109
|
+
"""The prompt body, HTML-flattened.
|
|
110
|
+
|
|
111
|
+
Prefer the dedicated `description` field. Fall back to `content` only when it
|
|
112
|
+
differs from the title, so a todo (where content == title) does not get its
|
|
113
|
+
own title echoed back as the prompt.
|
|
114
|
+
"""
|
|
115
|
+
desc = first_nonempty(record, ("description",))
|
|
116
|
+
if not desc:
|
|
117
|
+
content = first_nonempty(record, ("content",))
|
|
118
|
+
if content and content != title:
|
|
119
|
+
desc = content
|
|
120
|
+
return html_to_text(desc)
|
|
121
|
+
|
|
122
|
+
|
|
123
|
+
def make_id(title: str, keep_estimate: bool) -> str:
|
|
124
|
+
if keep_estimate:
|
|
125
|
+
return title
|
|
126
|
+
return ESTIMATE_SUFFIX_RE.sub("", title).strip()
|
|
127
|
+
|
|
128
|
+
|
|
129
|
+
def unwrap_items(payload: Any) -> list[dict[str, Any]]:
|
|
130
|
+
"""Accept the CLI envelope, the --agent raw form, a bare array, or one item."""
|
|
131
|
+
if isinstance(payload, dict):
|
|
132
|
+
if "data" in payload: # standard --json envelope: {ok, data, ...}
|
|
133
|
+
payload = payload["data"]
|
|
134
|
+
elif "tasks" in payload: # already a queue file
|
|
135
|
+
payload = payload["tasks"]
|
|
136
|
+
else: # a single recording object
|
|
137
|
+
return [payload]
|
|
138
|
+
if isinstance(payload, list):
|
|
139
|
+
return [item for item in payload if isinstance(item, dict)]
|
|
140
|
+
raise SystemExit("Unrecognized input JSON: expected an envelope, array, or object.")
|
|
141
|
+
|
|
142
|
+
|
|
143
|
+
def build_tasks(
|
|
144
|
+
items: list[dict[str, Any]],
|
|
145
|
+
*,
|
|
146
|
+
timeout_ms: int,
|
|
147
|
+
max_attempts: int,
|
|
148
|
+
keep_estimate: bool,
|
|
149
|
+
) -> tuple[list[dict[str, Any]], list[str]]:
|
|
150
|
+
tasks: list[dict[str, Any]] = []
|
|
151
|
+
warnings: list[str] = []
|
|
152
|
+
seen: dict[str, int] = {}
|
|
153
|
+
|
|
154
|
+
for index, record in enumerate(items):
|
|
155
|
+
title = extract_title(record)
|
|
156
|
+
if not title:
|
|
157
|
+
warnings.append(f"Item #{index} (id={record.get('id')}): no title; skipped.")
|
|
158
|
+
continue
|
|
159
|
+
|
|
160
|
+
prompt = extract_description(record, title)
|
|
161
|
+
if not prompt:
|
|
162
|
+
warnings.append(
|
|
163
|
+
f"Item #{index} {title!r}: empty description; skipped "
|
|
164
|
+
"(td-barrage requires a non-empty prompt)."
|
|
165
|
+
)
|
|
166
|
+
continue
|
|
167
|
+
|
|
168
|
+
task_id = make_id(title, keep_estimate)
|
|
169
|
+
if not task_id:
|
|
170
|
+
task_id = title # estimate-only title edge case
|
|
171
|
+
|
|
172
|
+
# De-duplicate ids so the file stays loadable (td-barrage rejects dupes).
|
|
173
|
+
if task_id in seen:
|
|
174
|
+
seen[task_id] += 1
|
|
175
|
+
deduped = f"{task_id} ({seen[task_id]})"
|
|
176
|
+
warnings.append(f"Duplicate id {task_id!r}; renamed to {deduped!r}.")
|
|
177
|
+
task_id = deduped
|
|
178
|
+
else:
|
|
179
|
+
seen[task_id] = 1
|
|
180
|
+
|
|
181
|
+
tasks.append(
|
|
182
|
+
{
|
|
183
|
+
"id": task_id,
|
|
184
|
+
"prompt": prompt,
|
|
185
|
+
"timeoutMs": timeout_ms,
|
|
186
|
+
"maxAttempts": max_attempts,
|
|
187
|
+
"success": {"strategy": "default"},
|
|
188
|
+
}
|
|
189
|
+
)
|
|
190
|
+
|
|
191
|
+
return tasks, warnings
|
|
192
|
+
|
|
193
|
+
|
|
194
|
+
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
|
|
195
|
+
parser = argparse.ArgumentParser(description=__doc__)
|
|
196
|
+
parser.add_argument(
|
|
197
|
+
"--input",
|
|
198
|
+
type=Path,
|
|
199
|
+
help="Path to the Basecamp CLI JSON. Reads stdin if omitted.",
|
|
200
|
+
)
|
|
201
|
+
parser.add_argument(
|
|
202
|
+
"--output",
|
|
203
|
+
type=Path,
|
|
204
|
+
default=SKILL_DIR / "queue.json",
|
|
205
|
+
help="Where to write the queue file (default: <skill>/queue.json).",
|
|
206
|
+
)
|
|
207
|
+
parser.add_argument(
|
|
208
|
+
"--timeout-mins",
|
|
209
|
+
type=float,
|
|
210
|
+
default=35.0,
|
|
211
|
+
help="Per-task timeout in minutes (default: 35).",
|
|
212
|
+
)
|
|
213
|
+
parser.add_argument(
|
|
214
|
+
"--max-attempts",
|
|
215
|
+
type=int,
|
|
216
|
+
default=2,
|
|
217
|
+
help="Per-task max attempts (default: 2).",
|
|
218
|
+
)
|
|
219
|
+
parser.add_argument(
|
|
220
|
+
"--keep-estimate",
|
|
221
|
+
action="store_true",
|
|
222
|
+
help="Keep the trailing [Nh] estimate suffix in the id (stripped by default).",
|
|
223
|
+
)
|
|
224
|
+
return parser.parse_args(argv)
|
|
225
|
+
|
|
226
|
+
|
|
227
|
+
def main(argv: list[str] | None = None) -> int:
|
|
228
|
+
args = parse_args(argv)
|
|
229
|
+
|
|
230
|
+
raw = args.input.read_text() if args.input else sys.stdin.read()
|
|
231
|
+
if not raw.strip():
|
|
232
|
+
print("No input JSON provided.", file=sys.stderr)
|
|
233
|
+
return 1
|
|
234
|
+
|
|
235
|
+
try:
|
|
236
|
+
payload = json.loads(raw)
|
|
237
|
+
except json.JSONDecodeError as exc:
|
|
238
|
+
print(f"Failed to parse input JSON: {exc}", file=sys.stderr)
|
|
239
|
+
return 1
|
|
240
|
+
|
|
241
|
+
if isinstance(payload, dict) and payload.get("ok") is False:
|
|
242
|
+
print(f"Basecamp CLI returned an error: {payload.get('error')}", file=sys.stderr)
|
|
243
|
+
return 1
|
|
244
|
+
|
|
245
|
+
items = unwrap_items(payload)
|
|
246
|
+
if not items:
|
|
247
|
+
print("Input contained no items.", file=sys.stderr)
|
|
248
|
+
return 1
|
|
249
|
+
|
|
250
|
+
timeout_ms = int(round(args.timeout_mins * 60 * 1000))
|
|
251
|
+
tasks, warnings = build_tasks(
|
|
252
|
+
items,
|
|
253
|
+
timeout_ms=timeout_ms,
|
|
254
|
+
max_attempts=args.max_attempts,
|
|
255
|
+
keep_estimate=args.keep_estimate,
|
|
256
|
+
)
|
|
257
|
+
|
|
258
|
+
for warning in warnings:
|
|
259
|
+
print(f"warning: {warning}", file=sys.stderr)
|
|
260
|
+
|
|
261
|
+
if not tasks:
|
|
262
|
+
print("No usable tasks produced (every item lacked a title or description).", file=sys.stderr)
|
|
263
|
+
return 1
|
|
264
|
+
|
|
265
|
+
args.output.parent.mkdir(parents=True, exist_ok=True)
|
|
266
|
+
args.output.write_text(json.dumps({"tasks": tasks}, indent=2) + "\n")
|
|
267
|
+
|
|
268
|
+
print(
|
|
269
|
+
f"Wrote {len(tasks)} task(s) to {args.output} "
|
|
270
|
+
f"(timeout={args.timeout_mins}m / {timeout_ms}ms, maxAttempts={args.max_attempts})."
|
|
271
|
+
)
|
|
272
|
+
return 0
|
|
273
|
+
|
|
274
|
+
|
|
275
|
+
if __name__ == "__main__":
|
|
276
|
+
raise SystemExit(main())
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
"""Tests for build_queue.py — verifies the Basecamp -> td-barrage mapping.
|
|
2
|
+
|
|
3
|
+
Run: python3 -m pytest .agents/skills/barrage/tests/ (or just `python3 test_build_queue.py`).
|
|
4
|
+
"""
|
|
5
|
+
|
|
6
|
+
from __future__ import annotations
|
|
7
|
+
|
|
8
|
+
import importlib.util
|
|
9
|
+
import json
|
|
10
|
+
import sys
|
|
11
|
+
from pathlib import Path
|
|
12
|
+
|
|
13
|
+
_SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "build_queue.py"
|
|
14
|
+
_spec = importlib.util.spec_from_file_location("build_queue", _SCRIPT)
|
|
15
|
+
assert _spec and _spec.loader
|
|
16
|
+
bq = importlib.util.module_from_spec(_spec)
|
|
17
|
+
_spec.loader.exec_module(bq)
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
# Shapes mirror what the Basecamp CLI actually returns (verified against the live
|
|
21
|
+
# CLI): todos expose title == content plus a separate description; cards carry the
|
|
22
|
+
# body in an HTML `content` field.
|
|
23
|
+
TODO = {
|
|
24
|
+
"id": 1,
|
|
25
|
+
"title": "Design — Search [2h]",
|
|
26
|
+
"content": "Design — Search [2h]",
|
|
27
|
+
"description": "Build the search UI and wire it to the API.",
|
|
28
|
+
}
|
|
29
|
+
CARD = {
|
|
30
|
+
"id": 2,
|
|
31
|
+
"title": "Implement parser [1.5h]",
|
|
32
|
+
"content": "<div>Parse the file.</div><div>Emit JSON.</div>",
|
|
33
|
+
"description": "",
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
def build(items, **kw):
|
|
38
|
+
defaults = {"timeout_ms": 2_100_000, "max_attempts": 2, "keep_estimate": False}
|
|
39
|
+
defaults.update(kw)
|
|
40
|
+
return bq.build_tasks(items, **defaults)
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
def test_todo_mapping():
|
|
44
|
+
tasks, warnings = build([TODO])
|
|
45
|
+
assert warnings == []
|
|
46
|
+
assert tasks == [
|
|
47
|
+
{
|
|
48
|
+
"id": "Design — Search", # [2h] stripped
|
|
49
|
+
"prompt": "Build the search UI and wire it to the API.",
|
|
50
|
+
"timeoutMs": 2_100_000,
|
|
51
|
+
"maxAttempts": 2,
|
|
52
|
+
"success": {"strategy": "default"},
|
|
53
|
+
}
|
|
54
|
+
]
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def test_card_html_body_becomes_prompt():
|
|
58
|
+
tasks, _ = build([CARD])
|
|
59
|
+
assert tasks[0]["id"] == "Implement parser"
|
|
60
|
+
# HTML flattened, block tags -> line breaks.
|
|
61
|
+
assert tasks[0]["prompt"] == "Parse the file.\nEmit JSON."
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def test_default_success_timeout_attempts():
|
|
65
|
+
tasks, _ = build([TODO], timeout_ms=999, max_attempts=5)
|
|
66
|
+
assert tasks[0]["success"] == {"strategy": "default"}
|
|
67
|
+
assert tasks[0]["timeoutMs"] == 999
|
|
68
|
+
assert tasks[0]["maxAttempts"] == 5
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
def test_keep_estimate_suffix():
|
|
72
|
+
tasks, _ = build([TODO], keep_estimate=True)
|
|
73
|
+
assert tasks[0]["id"] == "Design — Search [2h]"
|
|
74
|
+
|
|
75
|
+
|
|
76
|
+
def test_empty_description_is_skipped_with_warning():
|
|
77
|
+
item = {"id": 9, "title": "No body", "content": "No body", "description": ""}
|
|
78
|
+
tasks, warnings = build([item])
|
|
79
|
+
assert tasks == []
|
|
80
|
+
assert any("empty description" in w for w in warnings)
|
|
81
|
+
|
|
82
|
+
|
|
83
|
+
def test_todo_title_not_echoed_as_prompt():
|
|
84
|
+
# content == title and no description -> must NOT use title as the prompt.
|
|
85
|
+
item = {"id": 9, "title": "Echo", "content": "Echo", "description": ""}
|
|
86
|
+
tasks, _ = build([item])
|
|
87
|
+
assert tasks == []
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
def test_duplicate_ids_are_renamed():
|
|
91
|
+
a = {"id": 1, "title": "Same", "content": "Same", "description": "first"}
|
|
92
|
+
b = {"id": 2, "title": "Same", "content": "Same", "description": "second"}
|
|
93
|
+
tasks, warnings = build([a, b])
|
|
94
|
+
ids = [t["id"] for t in tasks]
|
|
95
|
+
assert ids == ["Same", "Same (2)"]
|
|
96
|
+
assert any("Duplicate" in w for w in warnings)
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
def test_unwrap_envelope_array_and_object():
|
|
100
|
+
assert bq.unwrap_items({"ok": True, "data": [TODO]}) == [TODO]
|
|
101
|
+
assert bq.unwrap_items([TODO]) == [TODO]
|
|
102
|
+
assert bq.unwrap_items(TODO) == [TODO]
|
|
103
|
+
assert bq.unwrap_items({"tasks": [TODO]}) == [TODO]
|
|
104
|
+
|
|
105
|
+
|
|
106
|
+
def test_end_to_end_via_main(tmp_path):
|
|
107
|
+
src = tmp_path / "items.json"
|
|
108
|
+
src.write_text(json.dumps({"ok": True, "data": [TODO, CARD]}))
|
|
109
|
+
out = tmp_path / "queue.json"
|
|
110
|
+
rc = bq.main(["--input", str(src), "--output", str(out)])
|
|
111
|
+
assert rc == 0
|
|
112
|
+
data = json.loads(out.read_text())
|
|
113
|
+
assert [t["id"] for t in data["tasks"]] == ["Design — Search", "Implement parser"]
|
|
114
|
+
assert all(t["success"] == {"strategy": "default"} for t in data["tasks"])
|
|
115
|
+
assert all(t["timeoutMs"] == 2_100_000 for t in data["tasks"])
|
|
116
|
+
|
|
117
|
+
|
|
118
|
+
if __name__ == "__main__":
|
|
119
|
+
failures = 0
|
|
120
|
+
for name, fn in sorted(globals().items()):
|
|
121
|
+
if name.startswith("test_") and callable(fn):
|
|
122
|
+
try:
|
|
123
|
+
# crude tmp_path for direct (non-pytest) runs
|
|
124
|
+
if "tmp_path" in fn.__code__.co_varnames[: fn.__code__.co_argcount]:
|
|
125
|
+
import tempfile
|
|
126
|
+
|
|
127
|
+
with tempfile.TemporaryDirectory() as d:
|
|
128
|
+
fn(Path(d))
|
|
129
|
+
else:
|
|
130
|
+
fn()
|
|
131
|
+
print(f"ok {name}")
|
|
132
|
+
except AssertionError as exc:
|
|
133
|
+
failures += 1
|
|
134
|
+
print(f"FAIL {name}: {exc}")
|
|
135
|
+
sys.exit(1 if failures else 0)
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: scry
|
|
3
|
+
version: 1.0.0
|
|
4
|
+
description: Single-site visual regression workflow for comparing a live URL against a preview/staging URL in Playwright UI mode. Use when the user provides or asks to compare two links, one live/baseline link and one preview link, wants fresh or reused live snapshots, or needs an ad hoc visual check that stores generated paths and baselines inside the skill folder rather than adding a normal CI site.
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Scry
|
|
8
|
+
|
|
9
|
+
Use `scry` for temporary visual regression checks between one live site and one preview site.
|
|
10
|
+
It keeps state in this skill folder so the workflow is separate from the repo's regular `sites/` configs.
|
|
11
|
+
|
|
12
|
+
## Workflow
|
|
13
|
+
|
|
14
|
+
1. Confirm the two URLs: live first, preview second.
|
|
15
|
+
2. Ask the user whether to refresh live baselines before comparing unless they already said yes/no.
|
|
16
|
+
3. Run the bundled script from the repository root:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
python3 .agents/skills/scry/scripts/scry.py <live-url> <preview-url> --refresh
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Use `--reuse` instead of `--refresh` when the user wants to compare against existing baselines.
|
|
23
|
+
The script opens Playwright UI mode by default.
|
|
24
|
+
|
|
25
|
+
## What The Script Does
|
|
26
|
+
|
|
27
|
+
- Writes `.agents/skills/scry/config.json` with the latest live and preview URLs.
|
|
28
|
+
- On first run, generates `.agents/skills/scry/paths.json` from the live URL's sitemap.
|
|
29
|
+
- Reuses `.agents/skills/scry/paths.json` on later runs unless `--regenerate-paths` is passed.
|
|
30
|
+
- Stores live baseline screenshots in `.agents/skills/scry/baselines/`.
|
|
31
|
+
- Generates runtime Playwright files under `.agents/skills/scry/runtime/`.
|
|
32
|
+
- Runs `npx playwright test --ui` against the generated runtime config.
|
|
33
|
+
|
|
34
|
+
If sitemap generation fails on the first run, the script falls back to a homepage-only `paths.json`.
|
|
35
|
+
After generation, inspect `paths.json` and trim duplicate template paths when the sitemap is large.
|
|
36
|
+
|
|
37
|
+
## Commands
|
|
38
|
+
|
|
39
|
+
Refresh baselines and open UI comparison:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
python3 .agents/skills/scry/scripts/scry.py https://example.com https://preview.example.com --refresh
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Reuse existing baselines:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
python3 .agents/skills/scry/scripts/scry.py https://example.com https://preview.example.com --reuse
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Regenerate paths from the live sitemap:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
python3 .agents/skills/scry/scripts/scry.py https://example.com https://preview.example.com --refresh --regenerate-paths
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Run headless instead of UI mode:
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
python3 .agents/skills/scry/scripts/scry.py https://example.com https://preview.example.com --refresh --no-ui
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Useful options:
|
|
64
|
+
|
|
65
|
+
- `--browser=chromium` limits the run to one browser. Repeat it for multiple browsers.
|
|
66
|
+
- `--max-diff=0.02` changes the allowed pixel ratio.
|
|
67
|
+
- `--sitemap=https://example.com/custom-sitemap.xml` uses a custom sitemap.
|
|
68
|
+
- `--path=/about --path=/products/example` overrides sitemap paths for the current run and writes them to `paths.json`.
|
|
69
|
+
- `--wait-until=networkidle` changes navigation waiting; default is `domcontentloaded`.
|
|
70
|
+
|
|
71
|
+
## Playwright CLI
|
|
72
|
+
|
|
73
|
+
If the comparison fails or needs manual investigation, use `$playwright-cli` after the UI run to inspect pages directly.
|
|
74
|
+
Prefer `--browser=chromium` or the existing Playwright CLI config in this repo when opening manual sessions.
|
|
@@ -0,0 +1,542 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Ad hoc visual regression runner for the scry skill.
|
|
3
|
+
|
|
4
|
+
The Python entrypoint owns URL/path discovery and writes the runtime Playwright
|
|
5
|
+
files used for comparison. Playwright still runs through `npx playwright test`
|
|
6
|
+
because the generated test suite is JavaScript and uses @playwright/test.
|
|
7
|
+
"""
|
|
8
|
+
|
|
9
|
+
from __future__ import annotations
|
|
10
|
+
|
|
11
|
+
import argparse
|
|
12
|
+
import html
|
|
13
|
+
import json
|
|
14
|
+
import os
|
|
15
|
+
import re
|
|
16
|
+
import subprocess
|
|
17
|
+
import sys
|
|
18
|
+
import urllib.error
|
|
19
|
+
import urllib.parse
|
|
20
|
+
import urllib.request
|
|
21
|
+
import xml.etree.ElementTree as ET
|
|
22
|
+
from datetime import datetime, timezone
|
|
23
|
+
from pathlib import Path, PurePosixPath
|
|
24
|
+
from typing import Any
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
SCRIPT_DIR = Path(__file__).resolve().parent
|
|
28
|
+
SKILL_DIR = SCRIPT_DIR.parent
|
|
29
|
+
PATHS_FILE = SKILL_DIR / "paths.json"
|
|
30
|
+
CONFIG_FILE = SKILL_DIR / "config.json"
|
|
31
|
+
BASELINE_DIR = SKILL_DIR / "baselines"
|
|
32
|
+
RUNTIME_DIR = SKILL_DIR / "runtime"
|
|
33
|
+
|
|
34
|
+
DEFAULT_BROWSERS = ("chromium", "firefox", "webkit")
|
|
35
|
+
WAIT_UNTIL_CHOICES = ("domcontentloaded", "networkidle")
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
PLAYWRIGHT_SPEC = r'''const { test, expect } = require("@playwright/test");
|
|
39
|
+
const fs = require("fs");
|
|
40
|
+
const path = require("path");
|
|
41
|
+
|
|
42
|
+
const config = JSON.parse(fs.readFileSync(process.env.SCRY_CONFIG, "utf8"));
|
|
43
|
+
const paths = JSON.parse(fs.readFileSync(process.env.SCRY_PATHS, "utf8"));
|
|
44
|
+
const refresh = process.env.SCRY_REFRESH === "1";
|
|
45
|
+
|
|
46
|
+
function joinUrl(baseUrl, pagePath) {
|
|
47
|
+
const base = new URL(baseUrl);
|
|
48
|
+
const normalizedPath = pagePath.startsWith("/") ? pagePath : "/" + pagePath;
|
|
49
|
+
const joined = new URL(normalizedPath, base.origin);
|
|
50
|
+
if (base.pathname !== "/" && base.pathname !== "") {
|
|
51
|
+
joined.pathname = path.posix.join(base.pathname, joined.pathname);
|
|
52
|
+
}
|
|
53
|
+
return joined.toString();
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
async function blockThirdPartyScripts(context, allowedUrl) {
|
|
57
|
+
const allowedHost = new URL(allowedUrl).hostname;
|
|
58
|
+
await context.route("**/*", async (route) => {
|
|
59
|
+
const request = route.request();
|
|
60
|
+
const resourceType = request.resourceType();
|
|
61
|
+
let requestHost = "";
|
|
62
|
+
try {
|
|
63
|
+
requestHost = new URL(request.url()).hostname;
|
|
64
|
+
} catch {
|
|
65
|
+
await route.continue();
|
|
66
|
+
return;
|
|
67
|
+
}
|
|
68
|
+
if (["script", "font", "media"].includes(resourceType) && requestHost !== allowedHost) {
|
|
69
|
+
await route.abort();
|
|
70
|
+
return;
|
|
71
|
+
}
|
|
72
|
+
await route.continue();
|
|
73
|
+
});
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
async function hidePopups(page) {
|
|
77
|
+
for (const selector of config.popupSelectors || []) {
|
|
78
|
+
await page.locator(selector).evaluateAll((elements) => {
|
|
79
|
+
for (const element of elements) {
|
|
80
|
+
element.style.setProperty("display", "none", "important");
|
|
81
|
+
element.style.setProperty("visibility", "hidden", "important");
|
|
82
|
+
}
|
|
83
|
+
}).catch(() => {});
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
async function freezeMedia(page) {
|
|
88
|
+
await page.addStyleTag({
|
|
89
|
+
content: "*,*::before,*::after{animation-delay:0s!important;animation-duration:0s!important;transition-duration:0s!important;scroll-behavior:auto!important} video,iframe{visibility:hidden!important}",
|
|
90
|
+
}).catch(() => {});
|
|
91
|
+
await page.evaluate(() => {
|
|
92
|
+
for (const video of document.querySelectorAll("video")) {
|
|
93
|
+
video.pause();
|
|
94
|
+
video.currentTime = 0;
|
|
95
|
+
}
|
|
96
|
+
}).catch(() => {});
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
async function waitForStability(page) {
|
|
100
|
+
for (const selector of config.readySelectors || []) {
|
|
101
|
+
await page.locator(selector).first().waitFor({
|
|
102
|
+
state: "visible",
|
|
103
|
+
timeout: config.settleTimeoutMs,
|
|
104
|
+
}).catch(() => {});
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
await page.evaluate(async () => {
|
|
108
|
+
if (document.fonts?.ready) await document.fonts.ready;
|
|
109
|
+
for (const img of document.images) {
|
|
110
|
+
img.loading = "eager";
|
|
111
|
+
img.fetchPriority = "high";
|
|
112
|
+
}
|
|
113
|
+
const height = Math.max(document.body.scrollHeight, document.documentElement.scrollHeight);
|
|
114
|
+
const step = Math.max(300, Math.floor(window.innerHeight * 0.8));
|
|
115
|
+
for (let y = 0; y < height; y += step) {
|
|
116
|
+
window.scrollTo(0, y);
|
|
117
|
+
await new Promise((resolve) => setTimeout(resolve, 80));
|
|
118
|
+
}
|
|
119
|
+
window.scrollTo(0, 0);
|
|
120
|
+
await Promise.all(Array.from(document.images).map((img) => {
|
|
121
|
+
if (img.complete && img.naturalWidth > 0) return Promise.resolve();
|
|
122
|
+
return new Promise((resolve) => {
|
|
123
|
+
img.addEventListener("load", resolve, { once: true });
|
|
124
|
+
img.addEventListener("error", resolve, { once: true });
|
|
125
|
+
setTimeout(resolve, 3000);
|
|
126
|
+
});
|
|
127
|
+
}));
|
|
128
|
+
}).catch(() => {});
|
|
129
|
+
|
|
130
|
+
await page.waitForTimeout(config.settleQuietWindowMs);
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
async function preparePage(page) {
|
|
134
|
+
await hidePopups(page);
|
|
135
|
+
await freezeMedia(page);
|
|
136
|
+
await waitForStability(page);
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
for (const pathEntry of paths) {
|
|
140
|
+
test(pathEntry.name + " visual comparison", async ({ browser }, testInfo) => {
|
|
141
|
+
const snapshotName = pathEntry.name + ".png";
|
|
142
|
+
const snapshotPath = testInfo.snapshotPath(snapshotName);
|
|
143
|
+
fs.mkdirSync(path.dirname(snapshotPath), { recursive: true });
|
|
144
|
+
|
|
145
|
+
if (refresh || !fs.existsSync(snapshotPath)) {
|
|
146
|
+
const liveContext = await browser.newContext({ reducedMotion: "reduce" });
|
|
147
|
+
await blockThirdPartyScripts(liveContext, config.liveUrl);
|
|
148
|
+
const livePage = await liveContext.newPage();
|
|
149
|
+
await livePage.goto(joinUrl(config.liveUrl, pathEntry.path), { waitUntil: config.waitUntil });
|
|
150
|
+
await preparePage(livePage);
|
|
151
|
+
const screenshot = await livePage.screenshot({
|
|
152
|
+
fullPage: true,
|
|
153
|
+
animations: "disabled",
|
|
154
|
+
scale: "css",
|
|
155
|
+
});
|
|
156
|
+
fs.writeFileSync(snapshotPath, screenshot);
|
|
157
|
+
await liveContext.close();
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
const previewContext = await browser.newContext({ reducedMotion: "reduce" });
|
|
161
|
+
await blockThirdPartyScripts(previewContext, config.previewUrl);
|
|
162
|
+
const previewPage = await previewContext.newPage();
|
|
163
|
+
await previewPage.goto(joinUrl(config.previewUrl, pathEntry.path), { waitUntil: config.waitUntil });
|
|
164
|
+
await preparePage(previewPage);
|
|
165
|
+
|
|
166
|
+
await expect(previewPage).toHaveScreenshot(snapshotName, {
|
|
167
|
+
fullPage: true,
|
|
168
|
+
animations: "disabled",
|
|
169
|
+
scale: "css",
|
|
170
|
+
maxDiffPixelRatio: config.maxDiffPixelRatio,
|
|
171
|
+
});
|
|
172
|
+
|
|
173
|
+
await previewContext.close();
|
|
174
|
+
});
|
|
175
|
+
}
|
|
176
|
+
'''
|
|
177
|
+
|
|
178
|
+
|
|
179
|
+
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
|
|
180
|
+
parser = argparse.ArgumentParser(
|
|
181
|
+
prog="scry.py",
|
|
182
|
+
formatter_class=argparse.RawDescriptionHelpFormatter,
|
|
183
|
+
description=(
|
|
184
|
+
"Compare a live URL against a preview URL with generated "
|
|
185
|
+
"Playwright visual regression tests."
|
|
186
|
+
),
|
|
187
|
+
)
|
|
188
|
+
parser.add_argument("live_url")
|
|
189
|
+
parser.add_argument("preview_url")
|
|
190
|
+
|
|
191
|
+
refresh_group = parser.add_mutually_exclusive_group()
|
|
192
|
+
refresh_group.add_argument(
|
|
193
|
+
"--refresh",
|
|
194
|
+
dest="refresh",
|
|
195
|
+
action="store_true",
|
|
196
|
+
default=None,
|
|
197
|
+
help="Capture new live baselines before comparing.",
|
|
198
|
+
)
|
|
199
|
+
refresh_group.add_argument(
|
|
200
|
+
"--reuse",
|
|
201
|
+
dest="refresh",
|
|
202
|
+
action="store_false",
|
|
203
|
+
help="Reuse existing live baselines.",
|
|
204
|
+
)
|
|
205
|
+
|
|
206
|
+
parser.add_argument(
|
|
207
|
+
"--regenerate-paths",
|
|
208
|
+
action="store_true",
|
|
209
|
+
help="Rebuild paths.json from the live sitemap.",
|
|
210
|
+
)
|
|
211
|
+
parser.add_argument(
|
|
212
|
+
"--path",
|
|
213
|
+
dest="explicit_paths",
|
|
214
|
+
action="append",
|
|
215
|
+
default=[],
|
|
216
|
+
help="Use an explicit path; repeat for multiple paths.",
|
|
217
|
+
)
|
|
218
|
+
parser.add_argument(
|
|
219
|
+
"--browser",
|
|
220
|
+
dest="browsers",
|
|
221
|
+
action="append",
|
|
222
|
+
default=[],
|
|
223
|
+
help="Limit browsers; repeat for multiple browsers.",
|
|
224
|
+
)
|
|
225
|
+
parser.add_argument("--sitemap", dest="sitemap_url", help="Use a custom sitemap URL.")
|
|
226
|
+
parser.add_argument(
|
|
227
|
+
"--max-diff",
|
|
228
|
+
type=float,
|
|
229
|
+
default=0.02,
|
|
230
|
+
help="Allowed visual diff ratio, from 0 to 1.",
|
|
231
|
+
)
|
|
232
|
+
parser.add_argument(
|
|
233
|
+
"--wait-until",
|
|
234
|
+
choices=WAIT_UNTIL_CHOICES,
|
|
235
|
+
default="domcontentloaded",
|
|
236
|
+
help="Navigation wait condition.",
|
|
237
|
+
)
|
|
238
|
+
parser.add_argument(
|
|
239
|
+
"--no-ui",
|
|
240
|
+
dest="ui",
|
|
241
|
+
action="store_false",
|
|
242
|
+
default=True,
|
|
243
|
+
help="Run headless instead of Playwright UI mode.",
|
|
244
|
+
)
|
|
245
|
+
|
|
246
|
+
args = parser.parse_args(argv)
|
|
247
|
+
if not 0 <= args.max_diff <= 1:
|
|
248
|
+
parser.error("--max-diff must be a number between 0 and 1.")
|
|
249
|
+
if not args.browsers:
|
|
250
|
+
args.browsers = list(DEFAULT_BROWSERS)
|
|
251
|
+
args.explicit_paths = [normalize_url_path(path) for path in args.explicit_paths]
|
|
252
|
+
return args
|
|
253
|
+
|
|
254
|
+
|
|
255
|
+
def normalize_url(value: str, label: str) -> str:
|
|
256
|
+
parsed = urllib.parse.urlsplit(value)
|
|
257
|
+
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
|
|
258
|
+
raise ValueError(f"{label} must be a valid http or https URL: {value}")
|
|
259
|
+
path = parsed.path or "/"
|
|
260
|
+
return urllib.parse.urlunsplit(
|
|
261
|
+
(parsed.scheme, parsed.netloc, path, parsed.query, parsed.fragment)
|
|
262
|
+
)
|
|
263
|
+
|
|
264
|
+
|
|
265
|
+
def normalize_url_path(value: str) -> str:
|
|
266
|
+
if not value or value == "/":
|
|
267
|
+
return "/"
|
|
268
|
+
parsed = urllib.parse.urlsplit(value)
|
|
269
|
+
path = parsed.path if parsed.scheme in {"http", "https"} else value
|
|
270
|
+
if not path or path == "/":
|
|
271
|
+
return "/"
|
|
272
|
+
with_slash = path if path.startswith("/") else f"/{path}"
|
|
273
|
+
return with_slash.rstrip("/") or "/"
|
|
274
|
+
|
|
275
|
+
|
|
276
|
+
def path_name(url_path: str) -> str:
|
|
277
|
+
if url_path == "/":
|
|
278
|
+
return "homepage"
|
|
279
|
+
name = "-".join(part for part in url_path.strip("/").split("/") if part)
|
|
280
|
+
name = re.sub(r"[^a-zA-Z0-9_-]+", "-", name)
|
|
281
|
+
name = re.sub(r"-+", "-", name)
|
|
282
|
+
return name.strip("-") or "homepage"
|
|
283
|
+
|
|
284
|
+
|
|
285
|
+
def default_sitemap_url(live_url: str) -> str:
|
|
286
|
+
return f"{live_url.rstrip('/')}/sitemap.xml"
|
|
287
|
+
|
|
288
|
+
|
|
289
|
+
def fetch_xml(url: str) -> str:
|
|
290
|
+
print(f"Fetching sitemap: {url}")
|
|
291
|
+
request = urllib.request.Request(
|
|
292
|
+
url,
|
|
293
|
+
headers={
|
|
294
|
+
"User-Agent": "ScryVisualRegression/1.0",
|
|
295
|
+
"Accept": "application/xml, text/xml, */*",
|
|
296
|
+
},
|
|
297
|
+
)
|
|
298
|
+
try:
|
|
299
|
+
with urllib.request.urlopen(request, timeout=30) as response:
|
|
300
|
+
charset = response.headers.get_content_charset() or "utf-8"
|
|
301
|
+
return response.read().decode(charset, errors="replace")
|
|
302
|
+
except urllib.error.HTTPError as exc:
|
|
303
|
+
raise RuntimeError(f"{exc.code} {exc.reason}") from exc
|
|
304
|
+
except urllib.error.URLError as exc:
|
|
305
|
+
raise RuntimeError(str(exc.reason)) from exc
|
|
306
|
+
|
|
307
|
+
|
|
308
|
+
def local_name(tag: str) -> str:
|
|
309
|
+
return tag.rsplit("}", 1)[-1].lower()
|
|
310
|
+
|
|
311
|
+
|
|
312
|
+
def extract_loc_values(xml: str) -> list[str]:
|
|
313
|
+
try:
|
|
314
|
+
root = ET.fromstring(xml)
|
|
315
|
+
except ET.ParseError:
|
|
316
|
+
return [
|
|
317
|
+
html.unescape(match.group(1).strip())
|
|
318
|
+
for match in re.finditer(r"<loc\b[^>]*>\s*([\s\S]*?)\s*</loc>", xml, re.I)
|
|
319
|
+
if match.group(1).strip()
|
|
320
|
+
]
|
|
321
|
+
|
|
322
|
+
values: list[str] = []
|
|
323
|
+
for element in root.iter():
|
|
324
|
+
if local_name(element.tag) == "loc" and element.text and element.text.strip():
|
|
325
|
+
values.append(element.text.strip())
|
|
326
|
+
return values
|
|
327
|
+
|
|
328
|
+
|
|
329
|
+
def is_sitemap_index(xml: str) -> bool:
|
|
330
|
+
try:
|
|
331
|
+
root = ET.fromstring(xml)
|
|
332
|
+
except ET.ParseError:
|
|
333
|
+
return bool(re.search(r"<sitemapindex\b", xml, re.I))
|
|
334
|
+
return local_name(root.tag) == "sitemapindex"
|
|
335
|
+
|
|
336
|
+
|
|
337
|
+
def parse_sitemap(
|
|
338
|
+
sitemap_url: str,
|
|
339
|
+
live_url: str,
|
|
340
|
+
seen: set[str] | None = None,
|
|
341
|
+
) -> list[dict[str, str]]:
|
|
342
|
+
seen = seen or set()
|
|
343
|
+
if sitemap_url in seen:
|
|
344
|
+
return []
|
|
345
|
+
seen.add(sitemap_url)
|
|
346
|
+
|
|
347
|
+
xml = fetch_xml(sitemap_url)
|
|
348
|
+
loc_values = extract_loc_values(xml)
|
|
349
|
+
|
|
350
|
+
if is_sitemap_index(xml):
|
|
351
|
+
paths: list[dict[str, str]] = []
|
|
352
|
+
for loc in loc_values:
|
|
353
|
+
paths.extend(parse_sitemap(loc, live_url, seen))
|
|
354
|
+
return paths
|
|
355
|
+
|
|
356
|
+
live_parts = urllib.parse.urlsplit(live_url)
|
|
357
|
+
live_origin = (live_parts.scheme, live_parts.netloc)
|
|
358
|
+
paths = []
|
|
359
|
+
for loc in loc_values:
|
|
360
|
+
url = urllib.parse.urlsplit(loc)
|
|
361
|
+
if (url.scheme, url.netloc) != live_origin:
|
|
362
|
+
continue
|
|
363
|
+
relative_path = normalize_url_path(url.path)
|
|
364
|
+
paths.append({"name": path_name(relative_path), "path": relative_path})
|
|
365
|
+
return paths
|
|
366
|
+
|
|
367
|
+
|
|
368
|
+
def unique_sorted_paths(paths: list[dict[str, str]]) -> list[dict[str, str]]:
|
|
369
|
+
by_path: dict[str, dict[str, str]] = {}
|
|
370
|
+
for entry in paths:
|
|
371
|
+
normalized = normalize_url_path(entry["path"])
|
|
372
|
+
if normalized not in by_path:
|
|
373
|
+
by_path[normalized] = {
|
|
374
|
+
"name": entry.get("name") or path_name(normalized),
|
|
375
|
+
"path": normalized,
|
|
376
|
+
}
|
|
377
|
+
return sorted(by_path.values(), key=lambda entry: entry["name"])
|
|
378
|
+
|
|
379
|
+
|
|
380
|
+
def write_json(path: Path, value: Any) -> None:
|
|
381
|
+
path.write_text(f"{json.dumps(value, indent=2)}\n", encoding="utf-8")
|
|
382
|
+
|
|
383
|
+
|
|
384
|
+
def read_json(path: Path) -> Any:
|
|
385
|
+
return json.loads(path.read_text(encoding="utf-8"))
|
|
386
|
+
|
|
387
|
+
|
|
388
|
+
def ensure_paths(live_url: str, args: argparse.Namespace) -> list[dict[str, str]]:
|
|
389
|
+
if args.explicit_paths:
|
|
390
|
+
paths = unique_sorted_paths(
|
|
391
|
+
[{"name": path_name(entry_path), "path": entry_path} for entry_path in args.explicit_paths]
|
|
392
|
+
)
|
|
393
|
+
write_json(PATHS_FILE, paths)
|
|
394
|
+
return paths
|
|
395
|
+
|
|
396
|
+
if PATHS_FILE.exists() and not args.regenerate_paths:
|
|
397
|
+
return read_json(PATHS_FILE)
|
|
398
|
+
|
|
399
|
+
sitemap_url = args.sitemap_url or default_sitemap_url(live_url)
|
|
400
|
+
try:
|
|
401
|
+
generated = unique_sorted_paths(parse_sitemap(sitemap_url, live_url))
|
|
402
|
+
if not generated:
|
|
403
|
+
raise RuntimeError("No same-origin URLs found in sitemap.")
|
|
404
|
+
write_json(PATHS_FILE, generated)
|
|
405
|
+
return generated
|
|
406
|
+
except Exception as exc:
|
|
407
|
+
if PATHS_FILE.exists():
|
|
408
|
+
raise
|
|
409
|
+
print(f"Could not generate sitemap paths ({exc}); using homepage only.", file=sys.stderr)
|
|
410
|
+
fallback = [{"name": "homepage", "path": "/"}]
|
|
411
|
+
write_json(PATHS_FILE, fallback)
|
|
412
|
+
return fallback
|
|
413
|
+
|
|
414
|
+
|
|
415
|
+
def resolve_refresh_choice(args: argparse.Namespace) -> bool:
|
|
416
|
+
if isinstance(args.refresh, bool):
|
|
417
|
+
return args.refresh
|
|
418
|
+
if not sys.stdin.isatty():
|
|
419
|
+
raise RuntimeError("Choose --refresh or --reuse. In non-interactive runs, scry will not guess.")
|
|
420
|
+
answer = input("Refresh live baselines before comparing? [Y/n] ")
|
|
421
|
+
return not re.match(r"^n(o)?$", answer.strip(), re.I)
|
|
422
|
+
|
|
423
|
+
|
|
424
|
+
def browser_project(browser: str) -> str:
|
|
425
|
+
device = {
|
|
426
|
+
"firefox": "Desktop Firefox",
|
|
427
|
+
"webkit": "Desktop Safari",
|
|
428
|
+
}.get(browser, "Desktop Chrome")
|
|
429
|
+
return f' {{ name: "{browser}", use: {{ ...devices["{device}"] }} }}'
|
|
430
|
+
|
|
431
|
+
|
|
432
|
+
def write_runtime_files(config: dict[str, Any]) -> Path:
|
|
433
|
+
RUNTIME_DIR.mkdir(parents=True, exist_ok=True)
|
|
434
|
+
|
|
435
|
+
spec_path = RUNTIME_DIR / "scry.spec.cjs"
|
|
436
|
+
playwright_config_path = RUNTIME_DIR / "playwright.config.cjs"
|
|
437
|
+
|
|
438
|
+
spec_path.write_text(PLAYWRIGHT_SPEC, encoding="utf-8")
|
|
439
|
+
browser_projects = ",\n".join(browser_project(browser) for browser in config["browsers"])
|
|
440
|
+
snapshot_dir = str(PurePosixPath(BASELINE_DIR))
|
|
441
|
+
playwright_config_path.write_text(
|
|
442
|
+
f'''const {{ defineConfig, devices }} = require("@playwright/test");
|
|
443
|
+
|
|
444
|
+
module.exports = defineConfig({{
|
|
445
|
+
testDir: __dirname,
|
|
446
|
+
timeout: 90000,
|
|
447
|
+
fullyParallel: true,
|
|
448
|
+
workers: 1,
|
|
449
|
+
reporter: "html",
|
|
450
|
+
use: {{
|
|
451
|
+
trace: "on-first-retry",
|
|
452
|
+
contextOptions: {{ reducedMotion: "reduce" }},
|
|
453
|
+
}},
|
|
454
|
+
snapshotDir: {json.dumps(snapshot_dir)},
|
|
455
|
+
snapshotPathTemplate: "{{snapshotDir}}/{{projectName}}/{{arg}}{{ext}}",
|
|
456
|
+
projects: [
|
|
457
|
+
{browser_projects}
|
|
458
|
+
],
|
|
459
|
+
}});
|
|
460
|
+
''',
|
|
461
|
+
encoding="utf-8",
|
|
462
|
+
)
|
|
463
|
+
return playwright_config_path
|
|
464
|
+
|
|
465
|
+
|
|
466
|
+
def run_playwright(playwright_config_path: Path, ui: bool, refresh: bool) -> int:
|
|
467
|
+
args = ["npx", "playwright", "test", "--config", str(playwright_config_path)]
|
|
468
|
+
if ui:
|
|
469
|
+
args.append("--ui")
|
|
470
|
+
|
|
471
|
+
env = os.environ.copy()
|
|
472
|
+
env.update(
|
|
473
|
+
{
|
|
474
|
+
"SCRY_CONFIG": str(CONFIG_FILE),
|
|
475
|
+
"SCRY_PATHS": str(PATHS_FILE),
|
|
476
|
+
"SCRY_BASELINE_DIR": str(BASELINE_DIR),
|
|
477
|
+
"SCRY_REFRESH": "1" if refresh else "0",
|
|
478
|
+
}
|
|
479
|
+
)
|
|
480
|
+
try:
|
|
481
|
+
completed = subprocess.run(args, env=env)
|
|
482
|
+
except FileNotFoundError as exc:
|
|
483
|
+
raise RuntimeError("npx was not found; install Node.js/npm before running scry.") from exc
|
|
484
|
+
return completed.returncode
|
|
485
|
+
|
|
486
|
+
|
|
487
|
+
def build_config(
|
|
488
|
+
live_url: str,
|
|
489
|
+
preview_url: str,
|
|
490
|
+
args: argparse.Namespace,
|
|
491
|
+
path_count: int,
|
|
492
|
+
) -> dict[str, Any]:
|
|
493
|
+
return {
|
|
494
|
+
"liveUrl": live_url,
|
|
495
|
+
"previewUrl": preview_url,
|
|
496
|
+
"sitemapUrl": args.sitemap_url or default_sitemap_url(live_url),
|
|
497
|
+
"waitUntil": args.wait_until,
|
|
498
|
+
"popupSelectors": [],
|
|
499
|
+
"readySelectors": ["main"],
|
|
500
|
+
"settleTimeoutMs": 10000,
|
|
501
|
+
"settleQuietWindowMs": 700,
|
|
502
|
+
"maxDiffPixelRatio": args.max_diff,
|
|
503
|
+
"browsers": args.browsers,
|
|
504
|
+
"pathCount": path_count,
|
|
505
|
+
"updatedAt": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
|
|
506
|
+
}
|
|
507
|
+
|
|
508
|
+
|
|
509
|
+
def main(argv: list[str] | None = None) -> int:
|
|
510
|
+
args = parse_args(argv)
|
|
511
|
+
live_url = normalize_url(args.live_url, "live URL")
|
|
512
|
+
preview_url = normalize_url(args.preview_url, "preview URL")
|
|
513
|
+
if args.sitemap_url:
|
|
514
|
+
args.sitemap_url = normalize_url(args.sitemap_url, "sitemap URL")
|
|
515
|
+
|
|
516
|
+
refresh = resolve_refresh_choice(args)
|
|
517
|
+
paths = ensure_paths(live_url, args)
|
|
518
|
+
|
|
519
|
+
BASELINE_DIR.mkdir(parents=True, exist_ok=True)
|
|
520
|
+
config = build_config(live_url, preview_url, args, len(paths))
|
|
521
|
+
write_json(CONFIG_FILE, config)
|
|
522
|
+
playwright_config_path = write_runtime_files(config)
|
|
523
|
+
|
|
524
|
+
print(f"Scry paths: {PATHS_FILE} ({len(paths)} paths)")
|
|
525
|
+
print(f"Scry baselines: {BASELINE_DIR}")
|
|
526
|
+
print(
|
|
527
|
+
"Refreshing live baselines during this run."
|
|
528
|
+
if refresh
|
|
529
|
+
else "Reusing existing live baselines."
|
|
530
|
+
)
|
|
531
|
+
|
|
532
|
+
return run_playwright(playwright_config_path, args.ui, refresh)
|
|
533
|
+
|
|
534
|
+
|
|
535
|
+
if __name__ == "__main__":
|
|
536
|
+
try:
|
|
537
|
+
sys.exit(main())
|
|
538
|
+
except KeyboardInterrupt:
|
|
539
|
+
sys.exit(130)
|
|
540
|
+
except Exception as exc:
|
|
541
|
+
print(f"scry: {exc}", file=sys.stderr)
|
|
542
|
+
sys.exit(1)
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
"""Tests for scry.py helper behavior.
|
|
2
|
+
|
|
3
|
+
Run: python3 skills/scry/tests/test_scry.py
|
|
4
|
+
"""
|
|
5
|
+
|
|
6
|
+
from __future__ import annotations
|
|
7
|
+
|
|
8
|
+
import importlib.util
|
|
9
|
+
import sys
|
|
10
|
+
from pathlib import Path
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
_SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "scry.py"
|
|
14
|
+
_spec = importlib.util.spec_from_file_location("scry", _SCRIPT)
|
|
15
|
+
assert _spec and _spec.loader
|
|
16
|
+
scry = importlib.util.module_from_spec(_spec)
|
|
17
|
+
_spec.loader.exec_module(scry)
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
def test_normalize_url_adds_origin_slash():
|
|
21
|
+
assert scry.normalize_url("https://example.com", "live URL") == "https://example.com/"
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
def test_normalize_url_rejects_non_http():
|
|
25
|
+
try:
|
|
26
|
+
scry.normalize_url("ftp://example.com", "live URL")
|
|
27
|
+
except ValueError as exc:
|
|
28
|
+
assert "http or https" in str(exc)
|
|
29
|
+
else:
|
|
30
|
+
raise AssertionError("Expected ValueError")
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
def test_normalize_url_path_and_name():
|
|
34
|
+
assert scry.normalize_url_path("about/team/") == "/about/team"
|
|
35
|
+
assert scry.normalize_url_path("https://example.com/products/widget/") == "/products/widget"
|
|
36
|
+
assert scry.path_name("/") == "homepage"
|
|
37
|
+
assert scry.path_name("/products/widget!") == "products-widget"
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
def test_unique_sorted_paths_dedupes_by_path():
|
|
41
|
+
paths = scry.unique_sorted_paths(
|
|
42
|
+
[
|
|
43
|
+
{"name": "z", "path": "/about/"},
|
|
44
|
+
{"name": "a", "path": "/"},
|
|
45
|
+
{"name": "duplicate", "path": "/about"},
|
|
46
|
+
]
|
|
47
|
+
)
|
|
48
|
+
assert paths == [
|
|
49
|
+
{"name": "a", "path": "/"},
|
|
50
|
+
{"name": "z", "path": "/about"},
|
|
51
|
+
]
|
|
52
|
+
|
|
53
|
+
|
|
54
|
+
def test_parse_sitemap_index_recurses_and_filters_origin():
|
|
55
|
+
fixtures = {
|
|
56
|
+
"https://example.com/sitemap.xml": """<?xml version="1.0"?>
|
|
57
|
+
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
|
58
|
+
<sitemap><loc>https://example.com/pages.xml</loc></sitemap>
|
|
59
|
+
</sitemapindex>""",
|
|
60
|
+
"https://example.com/pages.xml": """<?xml version="1.0"?>
|
|
61
|
+
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
|
62
|
+
<url><loc>https://example.com/about/</loc></url>
|
|
63
|
+
<url><loc>https://external.example.com/ignore/</loc></url>
|
|
64
|
+
</urlset>""",
|
|
65
|
+
}
|
|
66
|
+
original_fetch_xml = scry.fetch_xml
|
|
67
|
+
try:
|
|
68
|
+
scry.fetch_xml = lambda url: fixtures[url]
|
|
69
|
+
assert scry.parse_sitemap(
|
|
70
|
+
"https://example.com/sitemap.xml",
|
|
71
|
+
"https://example.com/",
|
|
72
|
+
) == [{"name": "about", "path": "/about"}]
|
|
73
|
+
finally:
|
|
74
|
+
scry.fetch_xml = original_fetch_xml
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
def test_parse_args_defaults_browsers():
|
|
78
|
+
args = scry.parse_args(["https://a.test", "https://b.test", "--reuse"])
|
|
79
|
+
assert args.refresh is False
|
|
80
|
+
assert args.browsers == ["chromium", "firefox", "webkit"]
|
|
81
|
+
|
|
82
|
+
|
|
83
|
+
if __name__ == "__main__":
|
|
84
|
+
failures = 0
|
|
85
|
+
for name, fn in sorted(globals().items()):
|
|
86
|
+
if name.startswith("test_") and callable(fn):
|
|
87
|
+
try:
|
|
88
|
+
fn()
|
|
89
|
+
print(f"ok {name}")
|
|
90
|
+
except AssertionError as exc:
|
|
91
|
+
failures += 1
|
|
92
|
+
print(f"FAIL {name}: {exc}")
|
|
93
|
+
except Exception as exc:
|
|
94
|
+
failures += 1
|
|
95
|
+
print(f"ERROR {name}: {exc}")
|
|
96
|
+
sys.exit(1 if failures else 0)
|