@jesdi/skills 0.0.32 → 0.0.34
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -5
- package/package.json +1 -1
- package/skills/implement-spec/ARCHITECTURE.md +4 -0
- package/skills/implement-spec/README.md +3 -1
- package/skills/implement-spec/SKILL.md +1 -1
- package/skills/implement-spec/driver.py +37 -4
- package/skills/implement-spec/driver_agents.py +44 -0
- package/skills/implement-spec/driver_final.py +6 -1
- package/skills/implement-spec/driver_run.py +16 -1
- package/skills/implement-spec/models.json +5 -5
- package/skills/implement-spec/tests/test_agents.py +35 -2
- package/skills/implement-spec/tests/test_driver.py +49 -3
- package/skills/implement-spec/tests/test_models.py +4 -4
- package/skills/pr-visual-diff/SKILL.md +85 -2
- package/skills/pr-visual-diff/references/capture-caller.yml +56 -0
- package/skills/pr-visual-diff/references/capture-spec-template.ts +1 -1
- package/skills-manifest.json +5 -5
package/README.md
CHANGED
|
@@ -144,11 +144,12 @@ them afterwards). `external-skills.json` lists them under `forks`.
|
|
|
144
144
|
|
|
145
145
|
### The box set
|
|
146
146
|
|
|
147
|
-
`external-skills.json` → `sets.box` names the skills the
|
|
148
|
-
receives
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
147
|
+
`external-skills.json` → `sets.box.external` names the third-party skills the
|
|
148
|
+
agent-ops box receives from the pinned upstream. `pr` is in this set and is
|
|
149
|
+
installed directly from mattpocock/skills. agent-ops-infra lists the own skills
|
|
150
|
+
of the box (`provision/my-skills.json`) and pins the commit of this file it
|
|
151
|
+
reads (`provision/external-skills.pin`); this repo's job is to publish the set
|
|
152
|
+
and keep the pins resolvable.
|
|
152
153
|
|
|
153
154
|
## Authoring skills
|
|
154
155
|
|
package/package.json
CHANGED
|
@@ -33,6 +33,10 @@ and do not know about the run, except `driver_run.py`, which the three deciding
|
|
|
33
33
|
- **A step that cannot go on raises `Blocked`.** The message says why. `with during("the merge"):` puts the step
|
|
34
34
|
in front. Only two places catch it for good: `driver._run_ticket` (the ticket becomes BLOCKED) and
|
|
35
35
|
`driver_final.run` (the end of run becomes BLOCKED). Another `except Blocked` only cleans up and raises again.
|
|
36
|
+
- **A CLI that fails with no report raises `Halted`**, not `Blocked`: the setup of the run is wrong, not the
|
|
37
|
+
ticket. `driver._run_ticket` and `driver._work` catch it: the run starts no other agent and ends with exit 4,
|
|
38
|
+
and the ticket keeps its phase for the next run. A cleanup of an `except Blocked` that leaves git as it was (an
|
|
39
|
+
open merge, an open rebase, a rejected fix batch) also runs for `Halted`.
|
|
36
40
|
- **A check before the first agent returns a list of errors**, so the operator sees all of them at one time
|
|
37
41
|
(exit 3). Examples: `precondition_errors`, `driver_tickets.parse`, `driver_report.validate`.
|
|
38
42
|
- **The modules that do not know about the run return a result, not `Blocked`.** Some return `""` or the reason
|
|
@@ -19,6 +19,7 @@ Before any agent starts, the driver checks these. Each failed check prints one l
|
|
|
19
19
|
- Each model entry is valid, and the CLI of the first entry of each role is on PATH. For the correctness role, the entry is the first of its list on another provider than the first implementer entry: that list must have such an entry, and its CLI must be on PATH. This applies to every run, also when no ticket has a **Risk** line.
|
|
20
20
|
- The `review-diff` skill is installed next to this one: the final review of every run reads it, and so does the correctness pass.
|
|
21
21
|
- With `--push` or `--unattended`, the repository has an `origin` remote.
|
|
22
|
+
- Each model that the run can use answers one short test prompt (the preflight). A model that fails gives one line with its error. `--dry-run` skips the preflight.
|
|
22
23
|
|
|
23
24
|
A run that stopped is continued with the same command. `state.json` is the only input of a resume.
|
|
24
25
|
|
|
@@ -56,6 +57,7 @@ Default timeouts in minutes: explorer 30, test-writer 30, implementer 90, review
|
|
|
56
57
|
| 1 | A ticket is BLOCKED, or the end of run is BLOCKED. The other tickets ran as far as they could. |
|
|
57
58
|
| 2 | Parked: a question of kind `locked-test` or `final-finding` is open. Exit 2 wins over exit 1. |
|
|
58
59
|
| 3 | Invalid input or a failed precondition: a usage error, a ticket error, a model error, an uncommitted change, a slot that cannot be made, another driver that runs in the same repository. No agent starts. The one exception: a review page that cannot be written gives exit 3 at the end of a run, after agents ran. |
|
|
60
|
+
| 4 | Stopped: the CLI of an agent failed two times and wrote no report (for example, the account cannot use the model). The driver starts no other agent and lets the running agents end. No ticket is BLOCKED by it: fix the CLI or the model entry, then run the same command again to resume. |
|
|
59
61
|
| 130 | The operator interrupted the run. The driver stops each agent. The work stays in the slots for the next run. |
|
|
60
62
|
|
|
61
63
|
With `--unattended`, the `--stage-json` file holds `{"stage": "implement", "status": …}`:
|
|
@@ -158,7 +160,7 @@ A repository with no `origin` remote skips only step 2 and step 6. The tidied co
|
|
|
158
160
|
|
|
159
161
|
The driver uses the first entry of a role. The fixer of a ticket runs on the `implementer` list; the `fixer` list is only for the fix batch of the final review. The driver uses a later entry in two cases only: fix rounds 3 and 4 (the implementer list), and the correctness pass, which takes the first correctness entry of another provider than the first implementer entry. The reviewer never does the correctness pass: when the list has no such entry, or the CLI of that entry is not on PATH, the run ends with exit 3 before any agent. The `explorer` list is read and checked, but the driver starts no explorer.
|
|
160
162
|
|
|
161
|
-
The defaults need both `claude` and `codex` on PATH. Read "Known limits" before you use them as they are.
|
|
163
|
+
The defaults need both `claude` and `codex` on PATH. The `openai` defaults name the full Codex model names (`gpt-6-astra`, `gpt-6.1-sol`), because a Codex CLI that is signed in with a ChatGPT account does not accept the short aliases. Read "Known limits" before you use them as they are.
|
|
162
164
|
|
|
163
165
|
## Report schema per role
|
|
164
166
|
|
|
@@ -24,7 +24,7 @@ has passed. The review rounds adapt superpowers' subagent-driven-development
|
|
|
24
24
|
Run the driver. It makes every choice of the run; you start it and relay its result.
|
|
25
25
|
|
|
26
26
|
1. In the PR worktree, run `sh <skill-dir>/run.sh` with the flags the operator gave, unchanged. `<skill-dir>` is the directory of this file. The driver needs `--gate` and `--spec`: ask the operator for a flag that is missing. A run can take hours: run the command in the background and wait for it to exit.
|
|
27
|
-
2. When it exits, relay `.agent/implement/summary.md` and the exit code: 0 is done, 1 is a BLOCKED ticket, 2 is parked on questions, 3 is invalid input. Relay the lines the driver printed after the summary too.
|
|
27
|
+
2. When it exits, relay `.agent/implement/summary.md` and the exit code: 0 is done, 1 is a BLOCKED ticket, 2 is parked on questions, 3 is invalid input, 4 is stopped by a CLI failure (after the fix, the same command resumes). Relay the lines the driver printed after the summary too.
|
|
28
28
|
3. With exit 2, and with exit 1 when the driver printed questions, show the operator those questions. When the operator answers, run the same command again with one `--answer <question id>=<option id>` for each answer.
|
|
29
29
|
|
|
30
30
|
Do not answer a question for the operator. Do not edit `.agent/implement/`, the code or the branches during a run.
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
[--dry-run]
|
|
7
7
|
|
|
8
8
|
Run it in the PR worktree. Exit 0 = every ticket merged (and pushed, when unattended), 1 = a ticket or the end of
|
|
9
|
-
run is BLOCKED, 2 = parked on a question, 3 = usage or validation error. README.md has the method and the files of
|
|
9
|
+
run is BLOCKED, 2 = parked on a question, 3 = usage or validation error, 4 = stopped by a CLI failure. README.md has the method and the files of
|
|
10
10
|
a run; ARCHITECTURE.md has the map of the modules and the rules the code keeps.
|
|
11
11
|
"""
|
|
12
12
|
from __future__ import annotations
|
|
@@ -32,7 +32,7 @@ import driver_slots
|
|
|
32
32
|
import driver_state
|
|
33
33
|
import driver_summary
|
|
34
34
|
import driver_tickets
|
|
35
|
-
from driver_run import FINAL, OUT, Blocked, Run, during, ledger
|
|
35
|
+
from driver_run import FINAL, OUT, Blocked, Halted, Run, during, ledger
|
|
36
36
|
from driver_slots import git
|
|
37
37
|
from driver_tickets import Ticket
|
|
38
38
|
|
|
@@ -133,7 +133,7 @@ def _resolve_conflict(ticket: Ticket, slot: str, run: Run, tip: str, files: list
|
|
|
133
133
|
changed = driver_review.locked_changes(slot, tip, run.state.read()["tickets"][number]["locked"])
|
|
134
134
|
if changed:
|
|
135
135
|
raise Blocked("; ".join(finding["title"] for finding in changed))
|
|
136
|
-
except Blocked:
|
|
136
|
+
except (Blocked, Halted):
|
|
137
137
|
driver_merge.abort(slot) # closes the merge when the agent did not commit it
|
|
138
138
|
raise
|
|
139
139
|
resolved = driver_slots.head(slot)
|
|
@@ -234,6 +234,12 @@ def _block(number: str, run: Run, ledger_line: int) -> None:
|
|
|
234
234
|
ledger(f"Ticket {number}: {driver_decisions.decides(ask.kind, len(ask.subjects))} in {DECISIONS}.")
|
|
235
235
|
|
|
236
236
|
|
|
237
|
+
def _halt(run: Run, owner: str, why: str) -> None:
|
|
238
|
+
"""Start no other agent: the run ends with exit 4, and `owner` keeps its phase for the next run."""
|
|
239
|
+
run.halted.append(why)
|
|
240
|
+
ledger(f"{owner}: the run stops and the phase stays for the next run: {why}.")
|
|
241
|
+
|
|
242
|
+
|
|
237
243
|
def _run_ticket(ticket: Ticket, slot_number: int, run: Run) -> bool:
|
|
238
244
|
"""Take one ticket from its recorded phase to merged or blocked. True: the slot is free again."""
|
|
239
245
|
number, slot = ticket.number, driver_slots.path(slot_number)
|
|
@@ -247,11 +253,15 @@ def _run_ticket(ticket: Ticket, slot_number: int, run: Run) -> bool:
|
|
|
247
253
|
except Blocked as blocked:
|
|
248
254
|
_block(number, run, ledger(f"Ticket {number}: BLOCKED at {blocked}."))
|
|
249
255
|
print(f"{ticket.file}: BLOCKED at {blocked}")
|
|
256
|
+
except Halted as halted:
|
|
257
|
+
_halt(run, f"Ticket {number}", str(halted))
|
|
250
258
|
return driver_slots.release(slot) # False: the slot keeps the uncommitted work of the blocked ticket
|
|
251
259
|
|
|
252
260
|
|
|
253
261
|
def _next_ticket(tickets: list[Ticket], running: dict, told: set, run: Run, kept: dict) -> Ticket | None:
|
|
254
262
|
"""The first ticket that may start now. A ticket whose slot is kept for it goes first: it needs no free slot."""
|
|
263
|
+
if run.halted:
|
|
264
|
+
return None
|
|
255
265
|
phases = {number: entry["phase"] for number, entry in run.state.read()["tickets"].items()}
|
|
256
266
|
ready, holds = driver_graph.frontier(tickets, phases, {number for number, _ in running.values()})
|
|
257
267
|
for hold in sorted(set(holds.items()) - told):
|
|
@@ -319,7 +329,13 @@ def _work(tickets: list[Ticket], slots: int, run: Run) -> tuple[bool, list[str]]
|
|
|
319
329
|
if driver_slots.release("."): # slot 1 is clean, so it goes back to the PR branch
|
|
320
330
|
git(".", "checkout", "-q", run.pr)
|
|
321
331
|
# Every ticket thread has ended, so every slot is idle: only now may the end of run rewrite the PR branch.
|
|
322
|
-
|
|
332
|
+
if only:
|
|
333
|
+
return False, []
|
|
334
|
+
try:
|
|
335
|
+
return driver_final.run(run, tickets)
|
|
336
|
+
except Halted as halted:
|
|
337
|
+
_halt(run, "End of run", str(halted))
|
|
338
|
+
return False, []
|
|
323
339
|
|
|
324
340
|
|
|
325
341
|
# ---------------------------------------------------------------- before the first agent: resume and answers
|
|
@@ -425,6 +441,17 @@ def _tell(args, code: int, open_questions: list[dict], blocked: list[str], pushe
|
|
|
425
441
|
_write_stage(args.stage_json, code, len(open_questions), blocked)
|
|
426
442
|
|
|
427
443
|
|
|
444
|
+
def _halted_exit(halted: list[str], final: dict) -> int:
|
|
445
|
+
"""Exit 4 after a CLI failure: no ticket is BLOCKED by it, and the same command resumes the run."""
|
|
446
|
+
for why in halted:
|
|
447
|
+
print(f"implement-spec: the run stopped: {why}")
|
|
448
|
+
print("implement-spec: fix the CLI or the model entry (--models FILE), then run the same command again to "
|
|
449
|
+
"resume; no ticket is BLOCKED by this.")
|
|
450
|
+
if error := _publish_questions(final["run"]):
|
|
451
|
+
print(error)
|
|
452
|
+
return 4
|
|
453
|
+
|
|
454
|
+
|
|
428
455
|
def _exit_code(final: dict, args, end_blocked: bool) -> int:
|
|
429
456
|
"""Publish the open questions, tell the operator or the dispatcher, and return the exit code."""
|
|
430
457
|
if error := _publish_questions(final["run"]):
|
|
@@ -456,6 +483,8 @@ def _run(tickets: list[Ticket], models: dict[str, list[str]], args, kept: dict[s
|
|
|
456
483
|
summary = driver_summary.render(final)
|
|
457
484
|
(OUT / "summary.md").write_text(summary)
|
|
458
485
|
print(summary + "".join(line + "\n" for line in lines), end="")
|
|
486
|
+
if run.halted:
|
|
487
|
+
return _halted_exit(run.halted, final)
|
|
459
488
|
return _exit_code(final, args, end_blocked)
|
|
460
489
|
|
|
461
490
|
|
|
@@ -572,6 +601,10 @@ def main(argv: list[str] | None = None) -> int:
|
|
|
572
601
|
if not (valid := _validated(args)):
|
|
573
602
|
return 3
|
|
574
603
|
tickets, models, kept = valid
|
|
604
|
+
# The CLIs answer before the first agent, so a model that the account cannot use stops no ticket.
|
|
605
|
+
if not args.runner and (errors := driver_agents.preflight(models)):
|
|
606
|
+
print("".join(error + "\n" for error in errors), end="")
|
|
607
|
+
return 3
|
|
575
608
|
_print_width(tickets)
|
|
576
609
|
return _run(tickets, models, args, kept)
|
|
577
610
|
|
|
@@ -11,8 +11,10 @@ import re
|
|
|
11
11
|
import shutil
|
|
12
12
|
import signal
|
|
13
13
|
import subprocess
|
|
14
|
+
import tempfile
|
|
14
15
|
import threading
|
|
15
16
|
import time
|
|
17
|
+
from concurrent.futures import ThreadPoolExecutor
|
|
16
18
|
from pathlib import Path
|
|
17
19
|
|
|
18
20
|
SKILL = Path(__file__).resolve().parent
|
|
@@ -22,6 +24,9 @@ TIMEOUTS = {"explorer": 30, "test-writer": 30, "implementer": 90, "reviewer": 45
|
|
|
22
24
|
PROVIDERS = {"anthropic": ("claude", ("low", "medium", "high", "xhigh", "max")),
|
|
23
25
|
"openai": ("codex", ("minimal", "low", "medium", "high", "xhigh"))}
|
|
24
26
|
READ_ONLY = ("reviewer", "correctness", "final-reviewer", "pr-writer")
|
|
27
|
+
PREFLIGHT = "implement-spec preflight: reply with the word OK. Read no file and run no command.\n"
|
|
28
|
+
# The time limit of one preflight call: a CLI that gives no answer in this time fails the preflight.
|
|
29
|
+
PREFLIGHT_MINUTES = 5
|
|
25
30
|
# A process group is here only while its leader is not reaped: then no other process can have its id.
|
|
26
31
|
LIVE: set[int] = set()
|
|
27
32
|
STOPPED = False
|
|
@@ -69,6 +74,45 @@ def load_models(path: str | None) -> tuple[dict[str, list[str]], list[str]]:
|
|
|
69
74
|
return models, [error for error in found if error]
|
|
70
75
|
|
|
71
76
|
|
|
77
|
+
def used_entries(models: dict[str, list[str]]) -> list[str]:
|
|
78
|
+
"""Each entry the driver can start an agent on: the first of each role but the explorer, the correctness entry
|
|
79
|
+
and the second implementer entry (fix rounds 3 and 4)."""
|
|
80
|
+
entries = [entries[0] for role, entries in models.items() if role not in ("explorer", "correctness")]
|
|
81
|
+
entries += [correctness_entry(models), *models["implementer"][1:2]]
|
|
82
|
+
return sorted(set(entries))
|
|
83
|
+
|
|
84
|
+
|
|
85
|
+
def preflight(models: dict[str, list[str]]) -> list[str]:
|
|
86
|
+
"""Give one short prompt to each used entry, all at the same time. Returns one error per entry whose CLI fails:
|
|
87
|
+
a model that the account cannot use, a CLI that is not signed in."""
|
|
88
|
+
with tempfile.TemporaryDirectory(prefix="implement-spec-preflight-") as folder:
|
|
89
|
+
def check(index: int, entry: str) -> str:
|
|
90
|
+
prompt, log = (Path(folder) / f"{index}{suffix}" for suffix in (".prompt.md", ".log"))
|
|
91
|
+
prompt.write_text(PREFLIGHT)
|
|
92
|
+
# A read-only command: the agent can change nothing, and codex writes its answer to the -o file.
|
|
93
|
+
code = launch(command("reviewer", entry, str(Path(folder) / f"{index}.json")), prompt, log,
|
|
94
|
+
PREFLIGHT_MINUTES * 60)
|
|
95
|
+
if code == 0:
|
|
96
|
+
return ""
|
|
97
|
+
why = f"no answer in {PREFLIGHT_MINUTES} minutes" if code is None else f"exit {code}"
|
|
98
|
+
cli = PROVIDERS[split(entry)[0]][0]
|
|
99
|
+
return f"model entry {entry}: the {cli} CLI failed the preflight ({why}): {last_words(log)}"
|
|
100
|
+
|
|
101
|
+
entries = used_entries(models)
|
|
102
|
+
with ThreadPoolExecutor(len(entries)) as pool:
|
|
103
|
+
return [error for error in pool.map(check, range(len(entries)), entries) if error]
|
|
104
|
+
|
|
105
|
+
|
|
106
|
+
def last_words(log: Path) -> str:
|
|
107
|
+
"""The last line of an agent log, or the `result` of the last stream-json line: the error a failed CLI gave."""
|
|
108
|
+
lines = [line for line in log.read_text(errors="replace").splitlines() if line.strip()] if log.exists() else []
|
|
109
|
+
if not lines:
|
|
110
|
+
return "no output"
|
|
111
|
+
with contextlib.suppress(ValueError, AttributeError):
|
|
112
|
+
return str(json.loads(lines[-1]).get("result") or lines[-1])[-500:]
|
|
113
|
+
return lines[-1][-500:]
|
|
114
|
+
|
|
115
|
+
|
|
72
116
|
def command(role: str, entry: str, report: str) -> list[str]:
|
|
73
117
|
"""The CLI call for one agent; the prompt goes to its stdin."""
|
|
74
118
|
provider, model, effort = split(entry)
|
|
@@ -29,7 +29,7 @@ import driver_decisions
|
|
|
29
29
|
import driver_merge
|
|
30
30
|
import driver_review
|
|
31
31
|
import driver_tidy
|
|
32
|
-
from driver_run import FINAL, OUT, Blocked, Run, during, ledger
|
|
32
|
+
from driver_run import FINAL, OUT, Blocked, Halted, Run, during, ledger
|
|
33
33
|
from driver_slots import git, head
|
|
34
34
|
from driver_state import State
|
|
35
35
|
from driver_tickets import Ticket
|
|
@@ -158,6 +158,9 @@ def _resolve_rebase_conflict(run: Run, env: dict, base: str, onto: str) -> None:
|
|
|
158
158
|
run.agent("conflict", FINAL, ".", values, template="rebase-conflict", detached=True)
|
|
159
159
|
except Blocked as blocked:
|
|
160
160
|
error = str(blocked)
|
|
161
|
+
except Halted:
|
|
162
|
+
git(".", "rebase", "--abort")
|
|
163
|
+
raise
|
|
161
164
|
if not error and _rebase_is_open(): # the agent resolved the files and did not go on
|
|
162
165
|
subprocess.run(["git", "-c", "core.editor=true", "rebase", "--continue"], capture_output=True, env=env)
|
|
163
166
|
if _rebase_is_open():
|
|
@@ -368,6 +371,8 @@ def _run_fixer(run: Run, tickets: list[Ticket], final: dict, tip: str) -> dict:
|
|
|
368
371
|
check={"owners": [ticket.number for ticket in tickets]})
|
|
369
372
|
except Blocked as blocked:
|
|
370
373
|
raise Blocked(f"{blocked}{_undo_rejected_batch(run, tip)}") from None
|
|
374
|
+
except Halted as halted:
|
|
375
|
+
raise Halted(f"{halted}{_undo_rejected_batch(run, tip)}") from None
|
|
371
376
|
|
|
372
377
|
|
|
373
378
|
def _fix_batch(run: Run, tickets: list[Ticket], final: dict, tip: str) -> None:
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
"""What the ticket threads and the end of run share: the Run, one agent with its one rerun, the gate, the ledger.
|
|
2
2
|
|
|
3
3
|
A step that cannot go on raises Blocked. The message says why, and `during` adds the step it happened in.
|
|
4
|
+
A CLI that fails with no report raises Halted: that is the setup of the run, not the ticket.
|
|
4
5
|
"""
|
|
5
6
|
from __future__ import annotations
|
|
6
7
|
|
|
@@ -34,6 +35,11 @@ class Blocked(Exception):
|
|
|
34
35
|
"""A ticket, or the end of run, cannot go on. The message says why."""
|
|
35
36
|
|
|
36
37
|
|
|
38
|
+
class Halted(Exception):
|
|
39
|
+
"""The CLI of an agent failed with no report, two times: a model the account cannot use, a CLI that is not
|
|
40
|
+
signed in. The run starts no other agent and ends with exit 4; the ticket keeps its phase for the next run."""
|
|
41
|
+
|
|
42
|
+
|
|
37
43
|
@contextmanager
|
|
38
44
|
def during(step: str) -> Iterator[None]:
|
|
39
45
|
"""Put the step in front of a Blocked that leaves the block: `<step>: <why>`."""
|
|
@@ -79,6 +85,7 @@ class Run:
|
|
|
79
85
|
kept: dict[str, int] # ticket number -> the slot that holds its uncommitted work from an earlier run
|
|
80
86
|
merging: threading.Lock = field(default_factory=threading.Lock) # held for one landing: merges are serial
|
|
81
87
|
asks: dict[str, Ask] = field(default_factory=dict) # ticket number -> its pending stop
|
|
88
|
+
halted: list[str] = field(default_factory=list) # why the run starts no other agent; [] while it goes on
|
|
82
89
|
|
|
83
90
|
def tip(self) -> str:
|
|
84
91
|
return git(".", "rev-parse", f"refs/heads/{self.pr}").stdout.strip()
|
|
@@ -110,6 +117,9 @@ class Run:
|
|
|
110
117
|
result = review()
|
|
111
118
|
except Blocked as blocked:
|
|
112
119
|
error = str(blocked)
|
|
120
|
+
except Halted:
|
|
121
|
+
_restore_if_changed(number, slot, branch, tip, runs_again=False)
|
|
122
|
+
raise
|
|
113
123
|
left = _restore_if_changed(number, slot, branch, tip, runs_again=not (error or last))
|
|
114
124
|
if error or not left:
|
|
115
125
|
break
|
|
@@ -244,7 +254,7 @@ def _attempts(run: Run, role: str, number: str, slot: str, values: dict[str, str
|
|
|
244
254
|
"template": template or role, **(check or {})}
|
|
245
255
|
argv = _command(run, role, entry, os.path.relpath(prompt, slot), report_in_slot)
|
|
246
256
|
text = _first_prompt(context["template"], values, notes, report_in_slot)
|
|
247
|
-
minutes, error = run.args.timeout[role], ""
|
|
257
|
+
minutes, error, failed = run.args.timeout[role], "", ""
|
|
248
258
|
for _ in range(2):
|
|
249
259
|
prompt.write_text(text)
|
|
250
260
|
report.unlink(missing_ok=True)
|
|
@@ -255,8 +265,13 @@ def _attempts(run: Run, role: str, number: str, slot: str, values: dict[str, str
|
|
|
255
265
|
_record_attempt(run, role, number, entry, seconds, log, error)
|
|
256
266
|
if not error:
|
|
257
267
|
return json.loads(report.read_text())
|
|
268
|
+
# A CLI that exits with an error and no report did not do the role: the run setup is wrong, not the ticket.
|
|
269
|
+
failed = "" if code in (None, 0) or report.exists() else (
|
|
270
|
+
f"the {role} on {entry} exited {code} with no report: {driver_agents.last_words(log)}")
|
|
258
271
|
# The rerun prompt is the first prompt, the rejection, then the same ending again.
|
|
259
272
|
text += rerun_lines + driver_agents.tail(text, report_in_slot)
|
|
260
273
|
if code is None:
|
|
261
274
|
minutes *= 2
|
|
275
|
+
if failed:
|
|
276
|
+
raise Halted(failed)
|
|
262
277
|
raise Blocked(error)
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
{
|
|
2
2
|
"explorer": ["claude-sonnet-5@medium"],
|
|
3
|
-
"test-writer": ["claude-sonnet-5@medium", "openai/gpt-sol@medium"],
|
|
4
|
-
"implementer": ["claude-sonnet-5@medium", "openai/gpt-sol@medium"],
|
|
5
|
-
"reviewer": ["openai/gpt-astra@medium", "claude-opus-5@medium"],
|
|
6
|
-
"correctness": ["openai/gpt-astra@medium", "claude-opus-5@medium"],
|
|
3
|
+
"test-writer": ["claude-sonnet-5@medium", "openai/gpt-6.1-sol@medium"],
|
|
4
|
+
"implementer": ["claude-sonnet-5@medium", "openai/gpt-6.1-sol@medium"],
|
|
5
|
+
"reviewer": ["openai/gpt-6-astra@medium", "claude-opus-5@medium"],
|
|
6
|
+
"correctness": ["openai/gpt-6-astra@medium", "claude-opus-5@medium"],
|
|
7
7
|
"fixer": ["claude-opus-5@medium"],
|
|
8
|
-
"final-reviewer": ["openai/gpt-astra@medium", "claude-opus-5@medium"],
|
|
8
|
+
"final-reviewer": ["openai/gpt-6-astra@medium", "claude-opus-5@medium"],
|
|
9
9
|
"conflict": ["claude-opus-5@medium"],
|
|
10
10
|
"pr-writer": ["claude-opus-5@medium"]
|
|
11
11
|
}
|
|
@@ -22,6 +22,7 @@ import pytest
|
|
|
22
22
|
from test_cli import GIT, ROLES, RUN, SKILL, TOOLS, drive, git, make_repo, models_file, ticket
|
|
23
23
|
|
|
24
24
|
RED = "FAILED tests/test_export.py::test_rows - AttributeError"
|
|
25
|
+
REFUSED = "ERROR: The 'gpt-astra' model is not supported when using Codex with a ChatGPT account."
|
|
25
26
|
RUNNER = f"""#!{sys.executable}
|
|
26
27
|
import json, os, subprocess, sys, time
|
|
27
28
|
from pathlib import Path
|
|
@@ -69,7 +70,8 @@ sys.exit(step.get("exit", 0))
|
|
|
69
70
|
|
|
70
71
|
@pytest.fixture
|
|
71
72
|
def env(tmp_path):
|
|
72
|
-
"""A restricted PATH whose fake claude and codex append their argv (NUL-separated) and exit 1.
|
|
73
|
+
"""A restricted PATH whose fake claude and codex append their argv (NUL-separated) and exit 1. They answer the
|
|
74
|
+
preflight prompt with exit 0 and log nothing, except for the model gpt-astra, which the account cannot use."""
|
|
73
75
|
bin_dir = tmp_path / "bin"
|
|
74
76
|
bin_dir.mkdir()
|
|
75
77
|
for name in TOOLS:
|
|
@@ -77,7 +79,15 @@ def env(tmp_path):
|
|
|
77
79
|
(bin_dir / name).symlink_to(shutil.which(name))
|
|
78
80
|
for cli in ("claude", "codex"):
|
|
79
81
|
log = f'"{tmp_path}/argv-{cli}"'
|
|
80
|
-
(bin_dir / cli).write_text(f"#!/bin/sh
|
|
82
|
+
(bin_dir / cli).write_text(f"""#!/bin/sh
|
|
83
|
+
case "$(cat)" in *'implement-spec preflight'*)
|
|
84
|
+
case " $* " in *' gpt-astra '*) echo "{REFUSED}"; exit 1;; esac
|
|
85
|
+
exit 0;;
|
|
86
|
+
esac
|
|
87
|
+
printf '%s\\0' "$@" >> {log}
|
|
88
|
+
printf '\\036' >> {log}
|
|
89
|
+
exit 1
|
|
90
|
+
""")
|
|
81
91
|
(bin_dir / cli).chmod(0o755)
|
|
82
92
|
return dict(os.environ, PATH=str(bin_dir)), tmp_path
|
|
83
93
|
|
|
@@ -267,6 +277,29 @@ def test_no_report_is_a_violation_with_one_rerun_then_blocked(env):
|
|
|
267
277
|
assert any("01" in l and "BLOCKED" in l for l in w.ledger().splitlines()), w.ledger()
|
|
268
278
|
|
|
269
279
|
|
|
280
|
+
def test_a_model_the_account_cannot_use_fails_the_preflight_with_exit_3_before_any_agent(env):
|
|
281
|
+
w = World(env, reviewer=["openai/gpt-astra@medium"])
|
|
282
|
+
proc = w.run({"test-writer": [tw()]}, runner=False)
|
|
283
|
+
assert proc.returncode == 3, proc.stdout + proc.stderr
|
|
284
|
+
assert f"model entry openai/gpt-astra@medium: the codex CLI failed the preflight (exit 1): {REFUSED}" \
|
|
285
|
+
in proc.stdout.splitlines(), proc.stdout
|
|
286
|
+
assert not launches(w.tmp, "claude") and not launches(w.tmp, "codex")
|
|
287
|
+
assert "ticket/01" not in git(w.repo, "branch", "--list") and not w.ledger()
|
|
288
|
+
|
|
289
|
+
|
|
290
|
+
def test_a_cli_that_fails_with_no_report_stops_the_run_with_exit_4_and_the_same_command_resumes(env):
|
|
291
|
+
w = World(env, refactor=True)
|
|
292
|
+
plan = {"implementer": [{"stderr": REFUSED + "\n", "exit": 1}] * 2 + [impl()]}
|
|
293
|
+
proc = w.run(plan)
|
|
294
|
+
assert proc.returncode == 4, proc.stdout + proc.stderr
|
|
295
|
+
assert f"implement-spec: the run stopped: the implementer on anthropic/claude-opus-5@medium exited 1 with no " \
|
|
296
|
+
f"report: {REFUSED}" in proc.stdout.splitlines(), proc.stdout
|
|
297
|
+
assert "BLOCKED" not in w.ledger() and "Ticket 01: the run stops and the phase stays for the next run: the implementer" in w.ledger()
|
|
298
|
+
assert json.loads((w.repo / ".agent/implement/state.json").read_text())["tickets"]["01"]["phase"] == "implement"
|
|
299
|
+
ok(w.run(plan))
|
|
300
|
+
assert len(w.calls("implementer")) == 3
|
|
301
|
+
|
|
302
|
+
|
|
270
303
|
def test_finished_status_reruns_the_implementer_once_with_the_error_appended(env):
|
|
271
304
|
w = World(env)
|
|
272
305
|
ok(w.run({"test-writer": [tw()], "implementer": [with_report(impl(0), status="FINISHED"), impl(1)]}))
|
|
@@ -175,7 +175,7 @@ def test_accepted_reports_end_the_run_with_exit_0(repo, tmp_path, capsys):
|
|
|
175
175
|
assert [line.partition(" for ")[0] for line in ledger] == [
|
|
176
176
|
"Ticket 01: test-writer ran on claude-sonnet-5@medium", "Ticket 01: implementer ran on claude-sonnet-5@medium",
|
|
177
177
|
f"Ticket 01: the gate is green on {tip}, record .agent/implement/01/gate-{tip}.log.",
|
|
178
|
-
"Ticket 01: reviewer ran on openai/gpt-astra@medium",
|
|
178
|
+
"Ticket 01: reviewer ran on openai/gpt-6-astra@medium",
|
|
179
179
|
f"Ticket 01: merged into feat/x as a fast-forward at {tip}."]
|
|
180
180
|
assert ledger[1].endswith(", cost 0.25 USD: report accepted.")
|
|
181
181
|
assert branches() == ["feat/x"] and "nothing to commit" in subprocess.run(
|
|
@@ -198,7 +198,7 @@ def prompt_of(tmp_path, role="implementer"):
|
|
|
198
198
|
|
|
199
199
|
def test_an_agent_that_writes_no_report_is_rerun_once_then_blocks(repo, tmp_path, capsys):
|
|
200
200
|
repo()
|
|
201
|
-
code, ledger = agents(tmp_path, capsys, "exit
|
|
201
|
+
code, ledger = agents(tmp_path, capsys, "exit 0\n", "--only", "reviewer")
|
|
202
202
|
assert code == 1 and len(ledger) == 3
|
|
203
203
|
assert ledger[2].startswith("Ticket 01: BLOCKED at the reviewer: report .agent/implement/01/reviewer-0.json is not")
|
|
204
204
|
|
|
@@ -422,7 +422,7 @@ def test_a_merge_conflict_goes_to_the_conflict_agent_once_and_its_commit_is_gate
|
|
|
422
422
|
|
|
423
423
|
def test_a_conflict_agent_that_does_not_resolve_blocks_the_ticket_and_keeps_its_branch(repo, tmp_path, capsys):
|
|
424
424
|
two(repo)
|
|
425
|
-
stuck = '[ $role != conflict ] || exit
|
|
425
|
+
stuck = '[ $role != conflict ] || exit 0\n'
|
|
426
426
|
code, ledger, out = walk(tmp_path, capsys, stuck + "FILE=work.txt\n" + WORK + TOGETHER + REPORT)
|
|
427
427
|
assert code == 1, out
|
|
428
428
|
(line,) = [line for line in ledger if "BLOCKED" in line]
|
|
@@ -436,6 +436,19 @@ def test_a_conflict_agent_that_does_not_resolve_blocks_the_ticket_and_keeps_its_
|
|
|
436
436
|
assert out_of("git", "-C", slot, "rev-parse", "-q", "--verify", "MERGE_HEAD") == ""
|
|
437
437
|
|
|
438
438
|
|
|
439
|
+
def test_a_conflict_cli_that_fails_stops_the_run_and_leaves_no_open_merge(repo, tmp_path, capsys):
|
|
440
|
+
two(repo)
|
|
441
|
+
failing = '[ $role != conflict ] || { echo "ERROR: not signed in"; exit 7; }\n'
|
|
442
|
+
code, ledger, out = walk(tmp_path, capsys, failing + "FILE=work.txt\n" + WORK + TOGETHER + REPORT)
|
|
443
|
+
assert code == 4 and "with no report: ERROR: not signed in" in out, out
|
|
444
|
+
assert not [line for line in ledger if "BLOCKED" in line]
|
|
445
|
+
(number,) = [n for n, entry in state_of(tmp_path)["tickets"].items() if entry["phase"] == "merge"]
|
|
446
|
+
assert f"ticket/{number}" in branches()
|
|
447
|
+
for slot in (".", ".agent/worktrees/slot-2"):
|
|
448
|
+
assert out_of("git", "-C", slot, "status", "--porcelain") == ""
|
|
449
|
+
assert out_of("git", "-C", slot, "rev-parse", "-q", "--verify", "MERGE_HEAD") == ""
|
|
450
|
+
|
|
451
|
+
|
|
439
452
|
def test_a_red_gate_blocks_the_ticket_and_the_ticket_it_blocks_never_starts(repo, tmp_path, capsys):
|
|
440
453
|
two(repo, second="01")
|
|
441
454
|
code, ledger, out = walk(tmp_path, capsys, WORK + REPORT, gate="echo CHECK-OUTPUT; exit 1")
|
|
@@ -765,3 +778,36 @@ def test_a_rerun_after_an_interrupt_keeps_the_work_in_each_slot_and_reruns_the_r
|
|
|
765
778
|
assert out_of("git", "show", "feat/x:wip-01.txt") == "half" and out_of("git", "show", "feat/x:wip-02.txt") == "half"
|
|
766
779
|
assert [line for line in ledger if "Ticket 02: resumed in slot 2 with the work the interrupted implementer left" in line]
|
|
767
780
|
assert out_of("git", "status", "--porcelain") == "" and branches() == ["feat/x"]
|
|
781
|
+
|
|
782
|
+
|
|
783
|
+
def test_the_preflight_asks_each_used_entry_once_and_names_each_cli_that_fails(tmp_path, monkeypatch):
|
|
784
|
+
"""claude answers; codex refuses gpt-astra, and the gpt-6.1-sol call never ends."""
|
|
785
|
+
bin_dir = tmp_path / "bin"
|
|
786
|
+
bin_dir.mkdir()
|
|
787
|
+
(bin_dir / "claude").write_text('#!/bin/sh\ncat > /dev/null\necho "$*" >> "$(dirname "$0")/asked"\n')
|
|
788
|
+
(bin_dir / "codex").write_text('#!/bin/sh\ncat > /dev/null\necho "$*" >> "$(dirname "$0")/asked"\n'
|
|
789
|
+
'case "$*" in *gpt-astra*) echo "ERROR: not supported"; exit 1;; '
|
|
790
|
+
'*gpt-6.1-sol*) sleep 30;; esac\n')
|
|
791
|
+
for cli in ("claude", "codex"):
|
|
792
|
+
(bin_dir / cli).chmod(0o755)
|
|
793
|
+
monkeypatch.setenv("PATH", f"{bin_dir}{os.pathsep}{os.environ['PATH']}")
|
|
794
|
+
monkeypatch.setattr(driver_agents, "PREFLIGHT_MINUTES", 0.02)
|
|
795
|
+
models, _ = driver_agents.load_models(None)
|
|
796
|
+
models["reviewer"] = ["openai/gpt-astra@medium"]
|
|
797
|
+
assert driver_agents.used_entries(models) == ["claude-opus-5@medium", "claude-sonnet-5@medium",
|
|
798
|
+
"openai/gpt-6-astra@medium", "openai/gpt-6.1-sol@medium",
|
|
799
|
+
"openai/gpt-astra@medium"]
|
|
800
|
+
assert driver_agents.preflight(models) == [
|
|
801
|
+
"model entry openai/gpt-6.1-sol@medium: the codex CLI failed the preflight (no answer in 0.02 minutes): "
|
|
802
|
+
"no output",
|
|
803
|
+
"model entry openai/gpt-astra@medium: the codex CLI failed the preflight (exit 1): ERROR: not supported"]
|
|
804
|
+
assert len((bin_dir / "asked").read_text().splitlines()) == 5
|
|
805
|
+
|
|
806
|
+
|
|
807
|
+
def test_last_words_takes_the_result_of_a_stream_json_line(tmp_path):
|
|
808
|
+
log = tmp_path / "agent.log"
|
|
809
|
+
log.write_text('{"type": "system"}\n{"type": "result", "is_error": true, "result": "There is an issue"}\n\n')
|
|
810
|
+
assert driver_agents.last_words(log) == "There is an issue"
|
|
811
|
+
log.write_text("ERROR: plain\n")
|
|
812
|
+
assert driver_agents.last_words(log) == "ERROR: plain"
|
|
813
|
+
assert driver_agents.last_words(tmp_path / "none.log") == "no output"
|
|
@@ -8,10 +8,10 @@ import pytest
|
|
|
8
8
|
from test_cli import RUN, SKILL, drive, env, make_repo, three # noqa: F401 (env is a fixture)
|
|
9
9
|
|
|
10
10
|
STANDARD = {
|
|
11
|
-
"implementer": ["claude-sonnet-5@medium", "openai/gpt-sol@medium"],
|
|
12
|
-
"test-writer": ["claude-sonnet-5@medium", "openai/gpt-sol@medium"],
|
|
13
|
-
"reviewer": ["openai/gpt-astra@medium", "claude-opus-5@medium"],
|
|
14
|
-
"final-reviewer": ["openai/gpt-astra@medium", "claude-opus-5@medium"],
|
|
11
|
+
"implementer": ["claude-sonnet-5@medium", "openai/gpt-6.1-sol@medium"],
|
|
12
|
+
"test-writer": ["claude-sonnet-5@medium", "openai/gpt-6.1-sol@medium"],
|
|
13
|
+
"reviewer": ["openai/gpt-6-astra@medium", "claude-opus-5@medium"],
|
|
14
|
+
"final-reviewer": ["openai/gpt-6-astra@medium", "claude-opus-5@medium"],
|
|
15
15
|
}
|
|
16
16
|
ENTRY = re.compile(r"^(?:(?:anthropic|openai)/)?[\w.-]+@(?:minimal|low|medium|high|xhigh|max)$")
|
|
17
17
|
|
|
@@ -45,7 +45,7 @@ file as JSON:
|
|
|
45
45
|
| `deviceProjects` | the config's `projects` (default to a desktop + a mobile project) |
|
|
46
46
|
| `baseBranch` | the repo's default branch (`main`/`master`; `git symbolic-ref refs/remotes/origin/HEAD`) |
|
|
47
47
|
| `packageManager` | infer from lockfile: `pnpm-lock.yaml`→pnpm, `package-lock.json`→npm, `yarn.lock`→yarn, `bun.lockb`→bun |
|
|
48
|
-
| `spinnerSelector` | the app
|
|
48
|
+
| `spinnerSelector` | every loading indicator of the app, as one selector list: spinners and skeletons (search for a spinner, `role="status"`, `aria-busy`, a skeleton component, `animate-pulse`), e.g. `.animate-spin, .animate-pulse`. A skeleton left out lets a half-loaded view pass as a good shot |
|
|
49
49
|
|
|
50
50
|
### Preconditions gate
|
|
51
51
|
|
|
@@ -77,6 +77,14 @@ compact report: the screenshot file names + pinned URLs, the PR-body
|
|
|
77
77
|
markdown, and any views it flagged. The main context creates/edits the PR from
|
|
78
78
|
that report — it never reads a PNG.
|
|
79
79
|
|
|
80
|
+
## Where the capture runs
|
|
81
|
+
|
|
82
|
+
- **Local** (the default): steps 3–5 below, on this machine.
|
|
83
|
+
- **Remote**: when the session instructions say so (the agent-ops box does),
|
|
84
|
+
or when this machine cannot start a Playwright browser. A GitHub workflow of
|
|
85
|
+
the target repo captures both sides and pushes the hidden ref; see "Remote
|
|
86
|
+
capture". Steps 1, 2 and 6 stay the same.
|
|
87
|
+
|
|
80
88
|
## Workflow
|
|
81
89
|
|
|
82
90
|
### 1. Map the diff to views, then get the user's go-ahead
|
|
@@ -96,6 +104,10 @@ hidden ref on the remote and embed them in the PR. Include the proposed plan: wh
|
|
|
96
104
|
device projects — default to the config's `deviceProjects`; add more only when
|
|
97
105
|
the change is layout-sensitive. If they decline, create the PR normally and stop.
|
|
98
106
|
|
|
107
|
+
In an unattended session (nobody answers questions, as on the agent-ops box),
|
|
108
|
+
do not ask: run the capture with the plan above and name the plan in the PR
|
|
109
|
+
body.
|
|
110
|
+
|
|
99
111
|
### 2. Write the capture spec
|
|
100
112
|
|
|
101
113
|
Copy `references/capture-spec-template.ts` from this skill into
|
|
@@ -133,7 +145,10 @@ That assertion is a backstop, not a substitute for looking: the subagent still
|
|
|
133
145
|
Reads each PNG after capture — an empty state, error toast, or wrong content
|
|
134
146
|
passes the spinner check but is still wrong. On a genuine failure, fix the spec
|
|
135
147
|
(add mock stubs, a wait for a stable element, or correct the spinner selector)
|
|
136
|
-
rather than shipping a bad screenshot.
|
|
148
|
+
rather than shipping a bad screenshot. A fixed element (a mobile bottom nav, a
|
|
149
|
+
sticky header) is drawn where the viewport was, so in a full-page shot it can
|
|
150
|
+
float over the middle of the page: give such a view `fullPage: false`, or one
|
|
151
|
+
shot per scroll position.
|
|
137
152
|
|
|
138
153
|
### 4. Capture "before" (merge base, temp worktree)
|
|
139
154
|
|
|
@@ -239,6 +254,74 @@ cell instead of an image:
|
|
|
239
254
|
| **Mobile** | _New view_ |  |
|
|
240
255
|
```
|
|
241
256
|
|
|
257
|
+
## Remote capture
|
|
258
|
+
|
|
259
|
+
Replaces steps 3–5. The subagent rule above still holds.
|
|
260
|
+
|
|
261
|
+
The steps run in one shared workflow, `.github/workflows/pr-visual-diff.yml` of
|
|
262
|
+
jesdi/general-skills. Each target repo has a short caller of it, made from
|
|
263
|
+
`references/capture-caller.yml` of this skill, which follows the main branch
|
|
264
|
+
of general-skills: GitHub starts `workflow_dispatch` only in the target repo.
|
|
265
|
+
|
|
266
|
+
**Precondition.** The default branch of the target repo has the caller,
|
|
267
|
+
`.github/workflows/pr-visual-diff.yml`:
|
|
268
|
+
|
|
269
|
+
```bash
|
|
270
|
+
REPO=$(gh repo view --json nameWithOwner -q .nameWithOwner)
|
|
271
|
+
gh api "repos/$REPO/contents/.github/workflows/pr-visual-diff.yml" --silent
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
If it does not, take no screenshots. Say in the PR body that the repo has no
|
|
275
|
+
pr-visual-diff workflow, and that `references/capture-caller.yml` of this
|
|
276
|
+
skill, filled from the discovery cache and merged to the default branch, adds
|
|
277
|
+
it. Do not add it in this PR: `workflow_dispatch` runs only from the default
|
|
278
|
+
branch.
|
|
279
|
+
|
|
280
|
+
**Dispatch.** Push the PR branch first: the workflow checks out commits of
|
|
281
|
+
origin. Write the filled spec of step 2 to a file, and a second copy whose
|
|
282
|
+
`VIEWS` holds only the views that exist at the merge base (none: no Before).
|
|
283
|
+
Then:
|
|
284
|
+
|
|
285
|
+
```bash
|
|
286
|
+
HEAD_SHA=$(git rev-parse HEAD)
|
|
287
|
+
BASE=$(git merge-base origin/<baseBranch> HEAD)
|
|
288
|
+
SLUG=$(git branch --show-current | tr / -)
|
|
289
|
+
CAPTURE="${HEAD_SHA:0:12}-$(date +%s)"
|
|
290
|
+
gh workflow run pr-visual-diff.yml --ref <baseBranch> \
|
|
291
|
+
-f capture="$CAPTURE" -f slug="$SLUG" -f head="$HEAD_SHA" -f base="$BASE" \
|
|
292
|
+
-f after_spec="$(base64 < <after-spec> | tr -d '\n')" \
|
|
293
|
+
-f before_spec="$(base64 < <before-spec> | tr -d '\n')" \
|
|
294
|
+
-f projects="<deviceProject>,<deviceProject>"
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
Leave `before_spec` empty when every view is new. `gh workflow run` prints no
|
|
298
|
+
run id: find the run by its name `pr-visual-diff $CAPTURE` (try for up to a
|
|
299
|
+
minute), then wait for it:
|
|
300
|
+
|
|
301
|
+
```bash
|
|
302
|
+
RUN=$(gh run list --workflow pr-visual-diff.yml --limit 20 --json databaseId,displayTitle \
|
|
303
|
+
--jq ".[] | select(.displayTitle == \"pr-visual-diff $CAPTURE\") | .databaseId")
|
|
304
|
+
gh run watch "$RUN" --exit-status
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
**A failed run.** Read `gh run view "$RUN" --log-failed | tail -80`. Fix the
|
|
308
|
+
spec as in step 3 (a mock stub, a wait, the spinner selector) and dispatch
|
|
309
|
+
once more with a new `CAPTURE`. When the second run fails too, take no
|
|
310
|
+
screenshots and say why in the PR body.
|
|
311
|
+
|
|
312
|
+
**Read the shots back.** Fetch the hidden ref and look at every PNG, as in
|
|
313
|
+
step 3:
|
|
314
|
+
|
|
315
|
+
```bash
|
|
316
|
+
git fetch origin "refs/pr-screenshots/$SLUG/$CAPTURE"
|
|
317
|
+
SHA=$(git rev-parse FETCH_HEAD)
|
|
318
|
+
mkdir -p <scratchpad>/shots && git archive "$SHA" | tar -x -C <scratchpad>/shots
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
A wrong shot is a failed run: fix the spec and dispatch again, under the same
|
|
322
|
+
limit. Then embed with step 6, with `$SHA` in the URLs. The ref stays: it keeps
|
|
323
|
+
the images alive.
|
|
324
|
+
|
|
242
325
|
## Adding shots to an existing PR
|
|
243
326
|
|
|
244
327
|
Same flow; the only differences: diff against the PR's base branch
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# pr-visual-diff remote capture of this repo. Template of the pr-visual-diff
|
|
2
|
+
# skill (references/capture-caller.yml): copy it to
|
|
3
|
+
# .github/workflows/pr-visual-diff.yml, replace every __PVD_*__ marker with the
|
|
4
|
+
# value of the discovery cache, and merge it to the default branch:
|
|
5
|
+
# workflow_dispatch runs only from there. The steps live in jesdi/general-skills
|
|
6
|
+
# (.github/workflows/pr-visual-diff.yml) and follow its main branch, like every
|
|
7
|
+
# jesdi skill follows its latest release.
|
|
8
|
+
|
|
9
|
+
name: pr-visual-diff
|
|
10
|
+
run-name: pr-visual-diff ${{ inputs.capture }}
|
|
11
|
+
|
|
12
|
+
on:
|
|
13
|
+
workflow_dispatch:
|
|
14
|
+
inputs:
|
|
15
|
+
capture:
|
|
16
|
+
description: 'Unique id of this capture; the last part of the hidden ref'
|
|
17
|
+
required: true
|
|
18
|
+
slug:
|
|
19
|
+
description: 'The PR branch with / as -'
|
|
20
|
+
required: true
|
|
21
|
+
head:
|
|
22
|
+
description: 'Commit captured as After'
|
|
23
|
+
required: true
|
|
24
|
+
base:
|
|
25
|
+
description: 'Commit captured as Before (the merge base)'
|
|
26
|
+
required: true
|
|
27
|
+
after_spec:
|
|
28
|
+
description: 'The filled capture spec, base64'
|
|
29
|
+
required: true
|
|
30
|
+
before_spec:
|
|
31
|
+
description: 'The filled capture spec of the views that exist at base, base64; empty: no Before'
|
|
32
|
+
required: false
|
|
33
|
+
default: ''
|
|
34
|
+
projects:
|
|
35
|
+
description: 'Playwright projects, separated by commas'
|
|
36
|
+
required: true
|
|
37
|
+
|
|
38
|
+
jobs:
|
|
39
|
+
capture:
|
|
40
|
+
permissions:
|
|
41
|
+
contents: write # only its publish job uses it, to push the hidden ref
|
|
42
|
+
uses: jesdi/general-skills/.github/workflows/pr-visual-diff.yml@main
|
|
43
|
+
with:
|
|
44
|
+
capture: ${{ inputs.capture }}
|
|
45
|
+
slug: ${{ inputs.slug }}
|
|
46
|
+
head: ${{ inputs.head }}
|
|
47
|
+
base: ${{ inputs.base }}
|
|
48
|
+
after_spec: ${{ inputs.after_spec }}
|
|
49
|
+
before_spec: ${{ inputs.before_spec }}
|
|
50
|
+
projects: ${{ inputs.projects }}
|
|
51
|
+
runner: __PVD_RUNNER__
|
|
52
|
+
frontend_dir: __PVD_FRONTEND_DIR__
|
|
53
|
+
specs_dir: __PVD_SPECS_DIR__
|
|
54
|
+
browsers: __PVD_BROWSERS__
|
|
55
|
+
node_version: '__PVD_NODE_VERSION__'
|
|
56
|
+
pnpm_version: '__PVD_PNPM_VERSION__'
|
|
@@ -26,7 +26,7 @@ const OUT = process.env.PR_SHOT_DIR;
|
|
|
26
26
|
const LABEL = process.env.PR_SHOT_LABEL ?? 'after';
|
|
27
27
|
if (!OUT) throw new Error('PR_SHOT_DIR must be set to an absolute output directory');
|
|
28
28
|
|
|
29
|
-
//
|
|
29
|
+
// Every loading indicator of this app, spinners and skeletons. From config: <spinnerSelector>.
|
|
30
30
|
const SPINNER = '__PVD_SPINNER_SELECTOR__';
|
|
31
31
|
|
|
32
32
|
// Pin device-pixel-ratio to 1 for every project. Device descriptors (iPhone 14,
|
package/skills-manifest.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"schemaVersion": 1,
|
|
3
|
-
"generatedAt": "2026-10-
|
|
3
|
+
"generatedAt": "2026-10-15T16:47:59.098Z",
|
|
4
4
|
"skills": [
|
|
5
5
|
{
|
|
6
6
|
"name": "backlog",
|
|
@@ -46,14 +46,14 @@
|
|
|
46
46
|
},
|
|
47
47
|
{
|
|
48
48
|
"name": "implement-spec",
|
|
49
|
-
"version": "0.1.
|
|
50
|
-
"hash": "sha256-
|
|
49
|
+
"version": "0.1.10",
|
|
50
|
+
"hash": "sha256-a0c42688cad43aece79d51d9e09644cc879518858781bf5563e6f8388d009a0c",
|
|
51
51
|
"description": "Implement a specification in code."
|
|
52
52
|
},
|
|
53
53
|
{
|
|
54
54
|
"name": "pr-visual-diff",
|
|
55
|
-
"version": "0.1.
|
|
56
|
-
"hash": "sha256-
|
|
55
|
+
"version": "0.1.2",
|
|
56
|
+
"hash": "sha256-df10fb424564e7721856a88a9bafeeb077041244faa949711ff92d5490fb7854",
|
|
57
57
|
"description": "Capture before/after screenshots of UI views with Playwright and embed them in a PR. Use this skill whenever creating or preparing a pull request that touches frontend code (components, views, styles, layout, Tailwind classes) — even if the user just says \"open a PR\" or \"create the PR\" after UI work, and especially if they mention screenshots, visual diff, before/after, or \"show what changed\". Also use when the user asks to add screenshots to an existing PR."
|
|
58
58
|
},
|
|
59
59
|
{
|