lastlight 0.7.4 → 0.7.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/config/default.yaml +1 -0
- package/dist/admin/auth.d.ts +25 -0
- package/dist/admin/auth.js +55 -10
- package/dist/admin/auth.js.map +1 -1
- package/dist/admin/routes.js +18 -1
- package/dist/admin/routes.js.map +1 -1
- package/dist/cli/cli-config.d.ts +9 -1
- package/dist/cli/cli-config.js +22 -0
- package/dist/cli/cli-config.js.map +1 -1
- package/dist/cli/cli.js +60 -4
- package/dist/cli/cli.js.map +1 -1
- package/dist/config/config.js +1 -0
- package/dist/config/config.js.map +1 -1
- package/dist/engine/dispatcher.js +19 -1
- package/dist/engine/dispatcher.js.map +1 -1
- package/dist/engine/executors/orchestrator.js +10 -1
- package/dist/engine/executors/orchestrator.js.map +1 -1
- package/dist/engine/github/github.d.ts +2 -0
- package/dist/engine/github/github.js +1 -0
- package/dist/engine/github/github.js.map +1 -1
- package/dist/engine/router.js +8 -4
- package/dist/engine/router.js.map +1 -1
- package/dist/engine/screen/classifier.js +8 -3
- package/dist/engine/screen/classifier.js.map +1 -1
- package/package.json +1 -1
- package/plugins/lastlight/.claude-plugin/plugin.json +1 -1
- package/plugins/lastlight/skills/lastlight-evals/SKILL.md +37 -7
- package/skills/README.md +8 -6
- package/skills/browser-qa/SKILL.md +15 -0
- package/skills/browser-qa/scripts/agent-browser.mjs +143 -7
- package/skills/code-review/SKILL.md +28 -5
- package/skills/demo/SKILL.md +10 -2
- package/skills/pr-review/SKILL.md +89 -33
- package/skills/pr-review/references/findings-schema.md +95 -0
- package/workflows/pr-review.yaml +150 -3
- package/workflows/prompts/demo.md +5 -2
- package/workflows/prompts/guardrails.md +15 -0
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: code-review
|
|
3
|
-
description: The shared rubric for reviewing a code change —
|
|
4
|
-
version:
|
|
3
|
+
description: The shared rubric for reviewing a code change — precision-first, high-signal findings only (Critical/Important), plus what to check (correctness, security, edge cases, regression risk, test coverage). Use when reviewing a PR or a branch diff.
|
|
4
|
+
version: 2.0.0
|
|
5
5
|
tags: [review, code-quality]
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -17,17 +17,40 @@ modified functions, check callers of anything whose signature or behaviour
|
|
|
17
17
|
changed for regression risk, and check that tests cover the actual risk areas,
|
|
18
18
|
not just the happy path.
|
|
19
19
|
|
|
20
|
+
## Precision first — post only what matters
|
|
21
|
+
|
|
22
|
+
A review is only useful if people trust it. A noisy reviewer gets muted; every
|
|
23
|
+
low-value comment you post spends the credibility of the ones that matter. So
|
|
24
|
+
the bar is **high-signal only**:
|
|
25
|
+
|
|
26
|
+
- **Post only Critical and Important findings.** Suggestions and Nits (below)
|
|
27
|
+
are *not* worth a formal review comment on their own — drop them, or fold at
|
|
28
|
+
most one genuinely valuable line into the summary. When in doubt, leave it out.
|
|
29
|
+
- **If you cannot name the concrete impact — what breaks, and for which input or
|
|
30
|
+
caller — do not post it.** "This could be cleaner" is not a finding; "this
|
|
31
|
+
crashes when `items` is empty because line 42 indexes `[0]`" is.
|
|
32
|
+
- **Confidence gate.** Before you finalise, re-read each finding against the
|
|
33
|
+
actual code and try to *refute your own claim*. If you can't defend it against
|
|
34
|
+
what the code actually does (not what you assumed), drop it. A missed marginal
|
|
35
|
+
issue costs far less than a wrong one.
|
|
36
|
+
|
|
20
37
|
## Finding tiers
|
|
21
38
|
|
|
22
|
-
Categorise every finding into exactly one tier
|
|
39
|
+
Categorise every finding into exactly one tier. **Only Critical and Important
|
|
40
|
+
are posted** (see Precision first):
|
|
23
41
|
|
|
24
42
|
- **Critical** — security issues, data loss, breaking changes, silent
|
|
25
43
|
data-dropping (see Correctness below). Blocks merge.
|
|
26
44
|
- **Important** — missing tests, performance problems, type errors, **avoidable
|
|
27
45
|
duplication**, **excessive complexity**, **compiler-silencing assertions**.
|
|
28
46
|
Should fix.
|
|
29
|
-
- **Suggestions** — clarity, naming, minor DRY tidy-ups.
|
|
30
|
-
|
|
47
|
+
- **Suggestions** — clarity, naming, minor DRY tidy-ups. *Not posted* — noise
|
|
48
|
+
in a formal review.
|
|
49
|
+
- **Nits** — style, formatting. *Not posted* — this is the linter's job.
|
|
50
|
+
|
|
51
|
+
Every posted finding carries a **one-line concrete impact**: the consequence
|
|
52
|
+
(what breaks / for whom) and, where it helps, the fix. That local reasoning is
|
|
53
|
+
what makes a comment actionable rather than a vague worry.
|
|
31
54
|
|
|
32
55
|
## What to check
|
|
33
56
|
|
package/skills/demo/SKILL.md
CHANGED
|
@@ -82,10 +82,18 @@ Author a `flow.json` (same shape as **browser-qa**) that scripts the
|
|
|
82
82
|
interaction as a *director*, not an operator:
|
|
83
83
|
|
|
84
84
|
- **Record the baseline first** — the starting state is act one.
|
|
85
|
-
- **Hold after state changes** — add a `
|
|
85
|
+
- **Hold after state changes** — add a `{"pause": 1200}` (or `waitFor`) so the
|
|
86
|
+
result is readable before moving on.
|
|
86
87
|
- **Verify between steps** — `waitFor` the expected element before the next
|
|
87
88
|
action; don't blindly fire keys into an unsettled page.
|
|
88
89
|
|
|
90
|
+
Because recording auto-enables the driver's **demo mode**, you get a visible
|
|
91
|
+
**synthetic cursor** that animates to each target, and holds between steps —
|
|
92
|
+
you don't script those. To make typing visible on screen, prefer **`type`**
|
|
93
|
+
(character-by-character) over `fill` (instant) for any input the viewer should
|
|
94
|
+
watch being filled. Tune pacing with `--step-delay` / `--type-delay` if a flow
|
|
95
|
+
feels rushed or draggy (see the browser-qa skill).
|
|
96
|
+
|
|
89
97
|
Record the session by passing `--record-dir`:
|
|
90
98
|
|
|
91
99
|
```bash
|
|
@@ -134,7 +142,7 @@ Composite the raw recording(s) into the final mp4 with the bundled wrapper:
|
|
|
134
142
|
--output <artifact-dir>/demo.mp4 \
|
|
135
143
|
--title "PR #42 — Add dark mode toggle" \
|
|
136
144
|
--subtitle "Toggling persists the theme across reloads" \
|
|
137
|
-
--layout single --speed 1
|
|
145
|
+
--layout single --speed 1 \
|
|
138
146
|
/tmp/demo-cap/session.webm
|
|
139
147
|
|
|
140
148
|
# Before/after comparison — first clip = left/BEFORE, second = right/AFTER
|
|
@@ -1,33 +1,42 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: pr-review
|
|
3
|
-
description: Review a GitHub pull request and post one formal review — advance the existing discussion
|
|
4
|
-
version:
|
|
3
|
+
description: Review a GitHub pull request and post one formal review — advance the existing discussion and give precision-first, high-signal feedback. A pure code review — no building. Use when asked to review a PR or on a cron PR scan.
|
|
4
|
+
version: 6.0.0
|
|
5
5
|
tags: [github, review, code-quality]
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# PR Review
|
|
9
9
|
|
|
10
|
-
Review an open PR
|
|
11
|
-
|
|
10
|
+
Review an open PR — high-signal findings only. This is a **pure code review**:
|
|
11
|
+
read the change and reason about it. Do **not** install dependencies, build, or
|
|
12
|
+
run tests — that is CI's job, and it validates whether the change actually works
|
|
13
|
+
far more reliably than you re-running it here. Your job is judgement on the diff,
|
|
14
|
+
not a build gate. A noisy review gets muted, so precision matters more than
|
|
15
|
+
volume.
|
|
12
16
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
17
|
+
You do **not** post the review yourself. You write your findings to a JSON file
|
|
18
|
+
(`.lastlight/pr-review/findings.json`) and a deterministic follow-up step posts
|
|
19
|
+
one formal review, anchoring each finding to its diff line as an inline comment
|
|
20
|
+
(§4).
|
|
21
|
+
|
|
22
|
+
This skill is the PR-specific procedure. It uses the **code-review** skill for
|
|
23
|
+
the precision bar and what-to-check rubric.
|
|
16
24
|
|
|
17
25
|
## Workspace
|
|
18
26
|
|
|
19
|
-
The harness pre-clones the PR's head ref
|
|
20
|
-
your cwd
|
|
21
|
-
|
|
27
|
+
The harness pre-clones the PR's head ref and drops you **inside the checkout** —
|
|
28
|
+
your cwd **is** the repo (`ls -la` shows `.git/` directly; `AGENTS.md` is the
|
|
29
|
+
sibling one level up at `../`). Use `git`/`read`/`grep` from here. To refresh:
|
|
22
30
|
`git fetch origin <branch> --depth 50 && git reset --hard FETCH_HEAD`. If the
|
|
23
|
-
|
|
31
|
+
checkout is somehow missing, `git clone https://github.com/{{owner}}/{{repo}}.git .`.
|
|
24
32
|
|
|
25
33
|
**Read code from this local checkout, never the API.** Use `git`/`read`/`grep`
|
|
26
34
|
on disk for the diff and file contents. Do **not** call
|
|
27
35
|
`github_get_pull_request_diff`, `github_list_pull_request_files`, or
|
|
28
36
|
`github_get_file_contents` — the API patch is a large redundant payload that
|
|
29
37
|
re-bloats context every turn. The `github_*` tools are for *API* operations only
|
|
30
|
-
(metadata
|
|
38
|
+
(reading metadata + prior comments in §1–2). You never post the review via a
|
|
39
|
+
tool — you write the findings file and the follow-up step posts it.
|
|
31
40
|
|
|
32
41
|
## Procedure
|
|
33
42
|
|
|
@@ -63,28 +72,75 @@ git diff --stat origin/<baseRef>...HEAD # churn
|
|
|
63
72
|
git diff origin/<baseRef>...HEAD # the patch
|
|
64
73
|
```
|
|
65
74
|
|
|
66
|
-
### 4.
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
-
|
|
75
|
+
### 4. Assess and write your findings
|
|
76
|
+
|
|
77
|
+
Apply the **code-review** skill's rubric — read each changed file in context;
|
|
78
|
+
check correctness / edge-cases / security / regression-risk / test-coverage.
|
|
79
|
+
Reason about the code statically; **don't build or run it** — trust CI to catch
|
|
80
|
+
what only running reveals, and spend your effort on what a human reviewer sees.
|
|
81
|
+
Follow that skill's **precision-first** rule: keep **only Critical and Important**
|
|
82
|
+
findings, each anchored to a `path:line` with a one-line concrete impact (what
|
|
83
|
+
breaks, for which input or caller). Drop Suggestions and Nits.
|
|
84
|
+
|
|
85
|
+
Before writing anything, run the **confidence gate**: re-read each finding
|
|
86
|
+
against the actual code and try to refute it; drop any you can't defend against
|
|
87
|
+
what the code really does. A clean PR should be approved with few or no
|
|
88
|
+
findings — that is a good review, not a lazy one.
|
|
89
|
+
|
|
90
|
+
**Do not call `github_create_pull_request_review` (or any review-submitting
|
|
91
|
+
tool).** Write your findings to `.lastlight/pr-review/findings.json` instead. A
|
|
92
|
+
deterministic follow-up step reads that file and posts one formal review with
|
|
93
|
+
your findings as inline comments anchored to the diff. The full contract with
|
|
94
|
+
worked examples is in [references/findings-schema.md](references/findings-schema.md);
|
|
95
|
+
the shape is:
|
|
96
|
+
|
|
97
|
+
```json
|
|
98
|
+
{
|
|
99
|
+
"skip": false,
|
|
100
|
+
"summary": "One or two sentences on what the PR does + overall assessment.",
|
|
101
|
+
"event": "COMMENT",
|
|
102
|
+
"base_ref": "main",
|
|
103
|
+
"head_sha": "<PR head SHA>",
|
|
104
|
+
"findings": [
|
|
105
|
+
{
|
|
106
|
+
"path": "src/foo.ts",
|
|
107
|
+
"line": 42,
|
|
108
|
+
"side": "RIGHT",
|
|
109
|
+
"severity": "Critical",
|
|
110
|
+
"title": "Short label for the finding",
|
|
111
|
+
"body": "Concrete impact — what breaks, for which input or caller.",
|
|
112
|
+
"suggestion": "exact replacement text for the anchored line(s)"
|
|
113
|
+
}
|
|
114
|
+
]
|
|
115
|
+
}
|
|
116
|
+
```
|
|
84
117
|
|
|
85
|
-
|
|
86
|
-
|
|
118
|
+
Rules:
|
|
119
|
+
- **Anchor precisely.** `path` must match the diff path exactly; `line`/`side`
|
|
120
|
+
must point at a line that appears in the diff (added/context → `side: RIGHT`;
|
|
121
|
+
removed/context → `side: LEFT`). A finding whose line isn't in the diff is
|
|
122
|
+
demoted to the summary body, so get the anchor right. Use optional `start_line`
|
|
123
|
+
(same side) for a multi-line range.
|
|
124
|
+
- `severity` is `Critical` or `Important` only.
|
|
125
|
+
- `suggestion` is optional — include it only when a concrete one-to-few-line fix
|
|
126
|
+
is obvious. It must be the exact replacement text for the anchored line(s),
|
|
127
|
+
nothing else; GitHub renders it as an applyable suggestion.
|
|
128
|
+
- `event` is `APPROVE` / `REQUEST_CHANGES` / `COMMENT`, matching what survived
|
|
129
|
+
the gate. A clean PR is an `APPROVE` with an empty `findings` array and a short
|
|
130
|
+
`summary`.
|
|
131
|
+
- `base_ref` and `head_sha` come from the `github_get_pull_request` call in §1 —
|
|
132
|
+
no extra API call. They are **mandatory**: without them the follow-up step
|
|
133
|
+
can't compute the diff and demotes every finding to the body.
|
|
134
|
+
- Create the dir and keep the file out of git first:
|
|
135
|
+
`mkdir -p .lastlight/pr-review && echo '.lastlight/' >> .git/info/exclude`.
|
|
136
|
+
|
|
137
|
+
**Stop / skip:** if a stop condition in §1 holds (bot-authored, merged, already
|
|
138
|
+
reviewed at the current head SHA), write `{"skip": true, "summary": "<reason>"}`
|
|
139
|
+
and stop — the follow-up step then posts nothing.
|
|
87
140
|
|
|
88
141
|
## Verification
|
|
89
142
|
|
|
90
|
-
Confirm
|
|
143
|
+
Confirm `.lastlight/pr-review/findings.json` is valid JSON, has `base_ref` +
|
|
144
|
+
`head_sha`, and every finding carries `path` + `line`. The deterministic
|
|
145
|
+
follow-up step posts the review and logs how many findings landed inline vs in
|
|
146
|
+
the body.
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# PR review findings schema
|
|
2
|
+
|
|
3
|
+
The `pr-review` skill writes its findings to `.lastlight/pr-review/findings.json`
|
|
4
|
+
(relative to the repo checkout — your cwd). The deterministic `post-review`
|
|
5
|
+
workflow phase reads this file and posts **one** formal GitHub review:
|
|
6
|
+
|
|
7
|
+
- Each finding whose `line`/`side` anchors to a line that appears in the PR diff
|
|
8
|
+
becomes an **inline comment** on that exact line.
|
|
9
|
+
- Any finding whose anchor isn't in the diff is **demoted** into the review body
|
|
10
|
+
under an "Additional findings" heading (GitHub rejects comments off the diff).
|
|
11
|
+
- If the diff can't be computed (missing `base_ref`, git failure), **all**
|
|
12
|
+
findings go into the body — the review still posts, so nothing is lost.
|
|
13
|
+
|
|
14
|
+
You never call `github_create_pull_request_review` yourself. Writing this file is
|
|
15
|
+
how you submit.
|
|
16
|
+
|
|
17
|
+
## Top-level object
|
|
18
|
+
|
|
19
|
+
| Field | Type | Required | Meaning |
|
|
20
|
+
|---|---|---|---|
|
|
21
|
+
| `skip` | boolean | no | `true` → you decided not to review (bot-authored / merged / already reviewed at head). The follow-up step posts nothing. |
|
|
22
|
+
| `summary` | string | yes | One or two sentences on what the PR does + your overall assessment. Becomes the review body. |
|
|
23
|
+
| `event` | string | yes | `APPROVE` \| `REQUEST_CHANGES` \| `COMMENT`. A clean PR is `APPROVE` with an empty `findings` array. |
|
|
24
|
+
| `base_ref` | string | yes | The PR base branch (e.g. `main`), from the `github_get_pull_request` call. Used to compute the diff for anchoring. |
|
|
25
|
+
| `head_sha` | string | yes | The PR head SHA, from the same call. Pins the review to the reviewed commit. |
|
|
26
|
+
| `findings` | array | yes | The surviving Critical/Important findings (may be empty). |
|
|
27
|
+
|
|
28
|
+
`base_ref` and `head_sha` are mandatory — without them the follow-up step can't
|
|
29
|
+
compute the diff and demotes every finding to the body.
|
|
30
|
+
|
|
31
|
+
## Finding object
|
|
32
|
+
|
|
33
|
+
| Field | Type | Required | Meaning |
|
|
34
|
+
|---|---|---|---|
|
|
35
|
+
| `path` | string | yes | Repo-relative file path, matching the diff path exactly. |
|
|
36
|
+
| `line` | number | yes | Line number on `side` that the comment anchors to. Must appear in the diff. |
|
|
37
|
+
| `side` | string | no | `RIGHT` (added/context line — default) or `LEFT` (removed/context line). |
|
|
38
|
+
| `start_line` | number | no | Start of a multi-line range (same `side` as `line`). |
|
|
39
|
+
| `severity` | string | yes | `Critical` or `Important` only. |
|
|
40
|
+
| `title` | string | yes | Short label for the finding. |
|
|
41
|
+
| `body` | string | yes | Concrete impact — what breaks, for which input or caller. |
|
|
42
|
+
| `suggestion` | string | no | Exact replacement text for the anchored line(s). Rendered as an applyable ```suggestion block. Include only when a concrete one-to-few-line fix is obvious. |
|
|
43
|
+
|
|
44
|
+
## Example — findings with an inline suggestion
|
|
45
|
+
|
|
46
|
+
```json
|
|
47
|
+
{
|
|
48
|
+
"skip": false,
|
|
49
|
+
"summary": "Adds a `--config` flag to the CLI and threads it into the connect path. Solid overall; one crash on the default path and one missing-await.",
|
|
50
|
+
"event": "REQUEST_CHANGES",
|
|
51
|
+
"base_ref": "main",
|
|
52
|
+
"head_sha": "9f3c1a2b7d4e5f60112233445566778899aabbcc",
|
|
53
|
+
"findings": [
|
|
54
|
+
{
|
|
55
|
+
"path": "src/cli.ts",
|
|
56
|
+
"line": 42,
|
|
57
|
+
"side": "RIGHT",
|
|
58
|
+
"severity": "Critical",
|
|
59
|
+
"title": "Null deref when --config is omitted",
|
|
60
|
+
"body": "`cfg.host` is undefined when no config file is passed, so every default-path invocation throws before connecting.",
|
|
61
|
+
"suggestion": "const host = cfg.host ?? DEFAULT_HOST;"
|
|
62
|
+
},
|
|
63
|
+
{
|
|
64
|
+
"path": "src/connect.ts",
|
|
65
|
+
"line": 88,
|
|
66
|
+
"side": "RIGHT",
|
|
67
|
+
"severity": "Important",
|
|
68
|
+
"title": "Missing await on disconnect()",
|
|
69
|
+
"body": "`disconnect()` returns a promise that's never awaited, so the socket can leak if the caller exits immediately after."
|
|
70
|
+
}
|
|
71
|
+
]
|
|
72
|
+
}
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Example — clean PR (approve, no findings)
|
|
76
|
+
|
|
77
|
+
```json
|
|
78
|
+
{
|
|
79
|
+
"skip": false,
|
|
80
|
+
"summary": "Small, well-tested refactor of the retry helper. No correctness or regression concerns.",
|
|
81
|
+
"event": "APPROVE",
|
|
82
|
+
"base_ref": "main",
|
|
83
|
+
"head_sha": "1122334455667788990011223344556677889900",
|
|
84
|
+
"findings": []
|
|
85
|
+
}
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
## Example — skip (already reviewed this SHA)
|
|
89
|
+
|
|
90
|
+
```json
|
|
91
|
+
{
|
|
92
|
+
"skip": true,
|
|
93
|
+
"summary": "A last-light[bot] review already exists on the current head SHA; nothing new to add."
|
|
94
|
+
}
|
|
95
|
+
```
|
package/workflows/pr-review.yaml
CHANGED
|
@@ -13,8 +13,155 @@ description: |
|
|
|
13
13
|
phases:
|
|
14
14
|
- name: review
|
|
15
15
|
label: Review
|
|
16
|
-
# pr-review: the PR procedure (primary).
|
|
17
|
-
#
|
|
18
|
-
|
|
16
|
+
# pr-review: the PR procedure (primary). code-review: the finding-tier +
|
|
17
|
+
# what-to-check rubric. No `building` — pr-review is a pure code review; CI
|
|
18
|
+
# validates that the change builds/runs.
|
|
19
|
+
#
|
|
20
|
+
# The agent does NOT submit the review itself — it writes structured
|
|
21
|
+
# findings to .lastlight/pr-review/findings.json (the "reference"). The
|
|
22
|
+
# deterministic `post-review` phase below reads that file and posts one
|
|
23
|
+
# formal review with the findings as line-anchored inline comments.
|
|
24
|
+
skills: [pr-review, code-review]
|
|
19
25
|
model: "{{models.review}}"
|
|
20
26
|
variant: "{{variants.review}}"
|
|
27
|
+
|
|
28
|
+
# Deterministic submission step: read the agent's findings.json and post ONE
|
|
29
|
+
# formal PR review, anchoring each finding to its diff line where possible and
|
|
30
|
+
# demoting off-diff findings to the summary body (GitHub 422s on off-diff
|
|
31
|
+
# lines). Terminal + always exits 0 — a posting hiccup must not fail an
|
|
32
|
+
# otherwise-good review (errors go to stderr, captured in the session log).
|
|
33
|
+
# owner/repo come from the template context; pr_number/base_ref/head_sha come
|
|
34
|
+
# from findings.json (the agent recorded them from github_get_pull_request).
|
|
35
|
+
# GITHUB_TOKEN is the run's minted review-write token; in evals the harness
|
|
36
|
+
# injects GITHUB_API_URL pointing at the mock so the review lands there.
|
|
37
|
+
- name: post-review
|
|
38
|
+
label: Post inline review
|
|
39
|
+
type: script
|
|
40
|
+
runtime: js
|
|
41
|
+
timeout_seconds: 120
|
|
42
|
+
script: |
|
|
43
|
+
import { readFileSync } from "node:fs";
|
|
44
|
+
import { execFileSync } from "node:child_process";
|
|
45
|
+
|
|
46
|
+
const api = (process.env.GITHUB_API_URL || "https://api.github.com").replace(/\/+$/, "");
|
|
47
|
+
const token = process.env.GITHUB_TOKEN || process.env.GH_TOKEN || "";
|
|
48
|
+
const owner = "{{owner}}";
|
|
49
|
+
const repo = "{{repo}}";
|
|
50
|
+
const log = (m) => process.stderr.write("[post-review] " + m + "\n");
|
|
51
|
+
|
|
52
|
+
let doc;
|
|
53
|
+
try {
|
|
54
|
+
doc = JSON.parse(readFileSync(".lastlight/pr-review/findings.json", "utf8"));
|
|
55
|
+
} catch (e) {
|
|
56
|
+
log("no readable findings.json (" + e.message + "); nothing to post");
|
|
57
|
+
process.exit(0);
|
|
58
|
+
}
|
|
59
|
+
if (doc.skip) { log("skip: " + (doc.summary || "agent skipped review")); process.exit(0); }
|
|
60
|
+
|
|
61
|
+
const prNumber = doc.pr_number;
|
|
62
|
+
const baseRef = doc.base_ref;
|
|
63
|
+
const headSha = doc.head_sha;
|
|
64
|
+
const findings = Array.isArray(doc.findings) ? doc.findings : [];
|
|
65
|
+
if (!prNumber) { log("findings.json missing pr_number; cannot post"); process.exit(0); }
|
|
66
|
+
|
|
67
|
+
// path -> Set of "SIDE:line". "+"/context are RIGHT:newLine; "-"/context
|
|
68
|
+
// are LEFT:oldLine. Mirrors GitHub's three-dot PR diff anchoring.
|
|
69
|
+
function parseDiff(diff) {
|
|
70
|
+
const map = new Map();
|
|
71
|
+
let path = null, right = 0, left = 0, inHunk = false;
|
|
72
|
+
for (const line of diff.split("\n")) {
|
|
73
|
+
if (line.startsWith("+++ ")) {
|
|
74
|
+
const p = line.slice(4).replace(/^b\//, "");
|
|
75
|
+
path = p === "/dev/null" ? null : p;
|
|
76
|
+
if (path && !map.has(path)) map.set(path, new Set());
|
|
77
|
+
inHunk = false;
|
|
78
|
+
} else if (line.startsWith("@@")) {
|
|
79
|
+
const m = /@@ -(\d+)(?:,\d+)? \+(\d+)(?:,\d+)? @@/.exec(line);
|
|
80
|
+
if (m) { left = parseInt(m[1], 10); right = parseInt(m[2], 10); inHunk = true; }
|
|
81
|
+
} else if (inHunk && path) {
|
|
82
|
+
const set = map.get(path);
|
|
83
|
+
if (line.startsWith("+")) { set.add("RIGHT:" + right); right++; }
|
|
84
|
+
else if (line.startsWith("-")) { set.add("LEFT:" + left); left++; }
|
|
85
|
+
else if (line.startsWith(" ")) { set.add("RIGHT:" + right); set.add("LEFT:" + left); right++; left++; }
|
|
86
|
+
else if (line.startsWith("\\")) { /* "" */ }
|
|
87
|
+
else { inHunk = false; }
|
|
88
|
+
}
|
|
89
|
+
}
|
|
90
|
+
return map;
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
// Commentable line set from the local checkout. Failure → null → demote all
|
|
94
|
+
// to the body (the review still posts; the judge/human still sees findings).
|
|
95
|
+
let commentable = null;
|
|
96
|
+
if (baseRef) {
|
|
97
|
+
try {
|
|
98
|
+
try { execFileSync("git", ["fetch", "origin", baseRef, "--depth", "50"], { stdio: "ignore" }); } catch {}
|
|
99
|
+
const diff = execFileSync("git", ["diff", "origin/" + baseRef + "...HEAD"], { encoding: "utf8", maxBuffer: 64 * 1024 * 1024 });
|
|
100
|
+
commentable = parseDiff(diff);
|
|
101
|
+
} catch (e) {
|
|
102
|
+
log("git diff failed (" + e.message + "); demoting all findings to the body");
|
|
103
|
+
}
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
const anchored = (f) => {
|
|
107
|
+
if (!commentable) return false;
|
|
108
|
+
const side = f.side === "LEFT" ? "LEFT" : "RIGHT";
|
|
109
|
+
const set = commentable.get(f.path);
|
|
110
|
+
if (!set || !set.has(side + ":" + f.line)) return false;
|
|
111
|
+
if (f.start_line && !set.has(side + ":" + f.start_line)) return false;
|
|
112
|
+
return true;
|
|
113
|
+
};
|
|
114
|
+
|
|
115
|
+
const inline = [], demoted = [];
|
|
116
|
+
for (const f of findings) {
|
|
117
|
+
if (f && f.path && f.line && anchored(f)) inline.push(f);
|
|
118
|
+
else if (f) demoted.push(f);
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
const FENCE = "```";
|
|
122
|
+
const commentBody = (f) => {
|
|
123
|
+
let b = "**[" + (f.severity || "Important") + "] " + (f.title || "") + "**\n\n" + (f.body || "");
|
|
124
|
+
if (f.suggestion) b += "\n\n" + FENCE + "suggestion\n" + f.suggestion + "\n" + FENCE;
|
|
125
|
+
return b;
|
|
126
|
+
};
|
|
127
|
+
const renderDemoted = (list) => list.length
|
|
128
|
+
? "\n\n### Additional findings\n" + list.map((f) =>
|
|
129
|
+
"- **[" + (f.severity || "Important") + "] " + (f.title || "") + "** (" + f.path + ":" + f.line + ") — " + (f.body || "")).join("\n")
|
|
130
|
+
: "";
|
|
131
|
+
const buildComments = (list) => list.map((f) => {
|
|
132
|
+
const side = f.side === "LEFT" ? "LEFT" : "RIGHT";
|
|
133
|
+
const c = { path: f.path, line: f.line, side, body: commentBody(f) };
|
|
134
|
+
if (f.start_line) { c.start_line = f.start_line; c.start_side = side; }
|
|
135
|
+
return c;
|
|
136
|
+
});
|
|
137
|
+
|
|
138
|
+
const event = doc.event || (findings.length === 0 ? "APPROVE" : "COMMENT");
|
|
139
|
+
async function postReview(comments, extraBody) {
|
|
140
|
+
const payload = { body: (doc.summary || "") + (extraBody || ""), event, comments };
|
|
141
|
+
if (headSha) payload.commit_id = headSha;
|
|
142
|
+
return fetch(api + "/repos/" + owner + "/" + repo + "/pulls/" + prNumber + "/reviews", {
|
|
143
|
+
method: "POST",
|
|
144
|
+
headers: {
|
|
145
|
+
"Authorization": "Bearer " + token,
|
|
146
|
+
"Accept": "application/vnd.github+json",
|
|
147
|
+
"Content-Type": "application/json",
|
|
148
|
+
"User-Agent": "last-light",
|
|
149
|
+
"X-GitHub-Api-Version": "2022-11-28",
|
|
150
|
+
},
|
|
151
|
+
body: JSON.stringify(payload),
|
|
152
|
+
});
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
let res = await postReview(buildComments(inline), renderDemoted(demoted));
|
|
156
|
+
if (!res.ok) {
|
|
157
|
+
const t = await res.text().catch(() => "");
|
|
158
|
+
log("inline review POST failed (" + res.status + "): " + t.slice(0, 300) + "; retrying body-only");
|
|
159
|
+
res = await postReview([], renderDemoted([...inline, ...demoted]));
|
|
160
|
+
if (!res.ok) {
|
|
161
|
+
const t2 = await res.text().catch(() => "");
|
|
162
|
+
log("body-only review POST also failed (" + res.status + "): " + t2.slice(0, 300));
|
|
163
|
+
process.exit(0);
|
|
164
|
+
}
|
|
165
|
+
}
|
|
166
|
+
log("posted review: " + inline.length + " inline, " + demoted.length + " in body, event=" + event);
|
|
167
|
+
process.exit(0);
|
|
@@ -76,7 +76,10 @@ wrong — STOP and fix it; do not ship a side-by-side that proves nothing.{{/if}
|
|
|
76
76
|
|
|
77
77
|
Author a `flow.json` (shape in the `browser-qa` skill) that scripts the
|
|
78
78
|
interaction like a director — record the baseline first, hold after state
|
|
79
|
-
changes, verify between steps.
|
|
79
|
+
changes (use `{"pause": 1200}` steps), verify between steps. Recording
|
|
80
|
+
auto-enables the driver's demo mode, so a **visible cursor** animates to each
|
|
81
|
+
target and a beat is held between steps — prefer `type` over `fill` so typing
|
|
82
|
+
shows on screen. Record with `--record-dir`:
|
|
80
83
|
|
|
81
84
|
```
|
|
82
85
|
node <browser-qa skill dir>/scripts/agent-browser.mjs run flow.json \
|
|
@@ -89,7 +92,7 @@ writing it into **`{{issueDir}}/demo.mp4`** (the harness harvests that dir):
|
|
|
89
92
|
```
|
|
90
93
|
<demo skill dir>/scripts/compose-demo.sh \
|
|
91
94
|
--output {{issueDir}}/demo.mp4 --title "<PR # — what it does>" \
|
|
92
|
-
--subtitle "<one line>" --layout single --speed 1
|
|
95
|
+
--subtitle "<one line>" --layout single --speed 1 --target-size-mb 5 \
|
|
93
96
|
/tmp/demo-cap/session.webm
|
|
94
97
|
```
|
|
95
98
|
|
|
@@ -10,6 +10,21 @@ THE ISSUE THIS BUILD WILL IMPLEMENT (use it to judge the escape hatch below):
|
|
|
10
10
|
SKIP CHECK — if {{issueDir}}/status.md already exists and contains
|
|
11
11
|
guardrails_status: READY, output "READY — guardrails already verified" and stop.
|
|
12
12
|
|
|
13
|
+
INSTALL DEPENDENCIES FIRST (do this before running ANY check command):
|
|
14
|
+
The harness pre-clones the repo but does NOT install dependencies, so the
|
|
15
|
+
test/lint/typecheck binaries (vitest, oxlint, tsc, eslint, ruff, …) will not
|
|
16
|
+
resolve until you install them. Detect the ecosystem and run the install:
|
|
17
|
+
- Node — pick the package manager from the lockfile: `pnpm-lock.yaml` → `pnpm install`,
|
|
18
|
+
`yarn.lock` → `yarn install`, `package-lock.json` (or none) → `npm ci`
|
|
19
|
+
(fall back to `npm install` if `npm ci` fails for a lockfile mismatch).
|
|
20
|
+
- Python — `pip install -e .` / `pip install -r requirements.txt`, or `poetry install` / `uv sync`.
|
|
21
|
+
- Rust — cargo fetches on first build; no separate step.
|
|
22
|
+
Only AFTER a successful install should you judge whether a command "runs". A
|
|
23
|
+
binary that is missing *after* dependencies installed cleanly is a real gap; a
|
|
24
|
+
binary missing *because deps were never installed* is NOT — do not BLOCK on it.
|
|
25
|
+
If the install itself fails (bad lockfile, missing manifest), that IS a blocking
|
|
26
|
+
guardrail — report it as such.
|
|
27
|
+
|
|
13
28
|
CHECK THESE GUARDRAILS:
|
|
14
29
|
|
|
15
30
|
1. **Test Framework** — Does the repo have a test runner (vitest, jest, pytest, cargo test, etc.)?
|