@jesdi/skills 0.0.36 → 0.0.38
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/skills/implement-spec/ARCHITECTURE.md +1 -1
- package/skills/implement-spec/README.md +5 -5
- package/skills/implement-spec/driver.py +8 -1
- package/skills/implement-spec/driver_decisions.py +4 -3
- package/skills/implement-spec/driver_fixloop.py +16 -15
- package/skills/implement-spec/driver_report.py +7 -0
- package/skills/implement-spec/driver_review.py +7 -4
- package/skills/implement-spec/driver_run.py +2 -0
- package/skills/implement-spec/prompts/correction.md +5 -3
- package/skills/implement-spec/prompts/correctness.md +1 -1
- package/skills/implement-spec/prompts/fixer.md +2 -1
- package/skills/implement-spec/prompts/implementer.md +1 -1
- package/skills/implement-spec/prompts/overruled.md +1 -1
- package/skills/implement-spec/prompts/reviewer.md +1 -1
- package/skills/implement-spec/prompts/test-writer.md +1 -1
- package/skills/implement-spec/tests/test_decisions.py +3 -2
- package/skills/implement-spec/tests/test_fixloop.py +71 -3
- package/skills-manifest.json +3 -3
package/package.json
CHANGED
|
@@ -12,7 +12,7 @@ A comment in the code is for a line that looks wrong or removable without it.
|
|
|
12
12
|
|---|---|
|
|
13
13
|
| `driver.py` | The command line, the checks before the first agent, the walk of the ticket graph, one ticket from its phase to merged or blocked, the landing on the PR branch, the exit code |
|
|
14
14
|
| `driver_run.py` | `Run` (what all ticket threads share), one agent with its one rerun, the gate call, the ledger, `Blocked` |
|
|
15
|
-
| `driver_fixloop.py` | A gated ticket tip in, a reviewed tip out: review,
|
|
15
|
+
| `driver_fixloop.py` | A gated ticket tip in, a reviewed tip out: review, test-writer correction, fix rounds |
|
|
16
16
|
| `driver_final.py` | The end of run: tidy, rebase, gate, final review, one fix batch, push |
|
|
17
17
|
| `driver_tickets.py`, `driver_graph.py` | Ticket files in, tickets or errors out; which tickets can start now |
|
|
18
18
|
| `driver_agents.py` | Model entries, the CLI command of each provider, prompt templates, the blocking launch |
|
|
@@ -100,7 +100,7 @@ A ticket is ready when each ticket in its **Blocked by** line is merged and no r
|
|
|
100
100
|
4. **Review.** The driver compares each locked file of the ticket with the test-writer commit. A file that differs is a critical finding `locked file changed: <path>`. Then the **reviewer** (`prompts/reviewer.md`) runs, and for a ticket with a **Risk** line the **correctness** pass (`prompts/correctness.md`) runs at the same time in the same slot.
|
|
101
101
|
5. **Merge.** The driver merges the reviewed tip into the PR branch with git plumbing, then deletes the ticket branch. A conflict goes to the **conflict** agent (`prompts/conflict.md`) once. Its resolution may change no locked file of the ticket. A second conflict blocks the ticket.
|
|
102
102
|
**The landing check.** Merges are serial. Before the PR branch moves, the driver compares the tree that is about to land (the ticket tip of a fast-forward, or the merge commit) with the PR tip: each locked file of each merged ticket must be the same tree entry in both: the same blob with the same mode, so a file that became a symlink is a difference. A file that only one of them has is a difference. A file that the landing ticket lists in its own `locked` is not checked: the test-writer of a later ticket can add a test to a file that an earlier ticket locked, and must list that file. A test-writer report that changes such a file and does not list it is rejected, with the usual one rerun. A difference refuses the landing: the ticket is BLOCKED with `locked file changed: <path>` in the ledger, the PR branch does not move, and no fix round runs. The PR branch moves only from the tip that the check read; when another hand moved the branch in between, the driver makes the merge, the check and the gate again, one time. Only that case starts the landing again: a refused landing and a red merge gate block the ticket. The second try runs no conflict agent, so a conflict there blocks the ticket, and so does a branch that moved again.
|
|
103
|
-
**Before a correction.** A test-writer that corrects locked tests (after `overrule-test`, or for a fixture finding) starts from the locked versions: the driver puts back, in the tree, each locked file of the ticket and each locked file of a merged ticket that the ticket changed, and the ledger names each file. A file of a merged ticket goes back to the version of the PR branch.
|
|
103
|
+
**Before a correction.** A test-writer that corrects locked tests (after `overrule-test`, or for a `fixture` finding or a `test` finding on a locked file) starts from the locked versions: the driver puts back, in the tree, each locked file of the ticket and each locked file of a merged ticket that the ticket changed, and the ledger names each file. A file of a merged ticket goes back to the version of the PR branch.
|
|
104
104
|
|
|
105
105
|
Rules for each agent:
|
|
106
106
|
|
|
@@ -113,13 +113,13 @@ Rules for each agent:
|
|
|
113
113
|
|
|
114
114
|
| Finding | What the driver does |
|
|
115
115
|
|---|---|
|
|
116
|
-
| kind `fixture`, any severity | One test-writer correction, outside the fix rounds. The next review is a full review. |
|
|
116
|
+
| kind `fixture`, any severity, or kind `test`, `critical` or `important`, on a locked file of the ticket | One test-writer correction, outside the fix rounds. The fixer may not edit a locked file, so a fix round cannot close a `test` finding there. The corrected files are the new locked versions, and the next review is a full review. A finding of this kind that is open after the correction blocks the ticket. |
|
|
117
117
|
| `critical` or `important` | One fix round. |
|
|
118
118
|
| `minor` | It stays in the ledger and in the findings file. The final reviewer gets the list. |
|
|
119
119
|
|
|
120
120
|
A fix round is one **fixer** (`prompts/fixer.md`), then the gate on the new tip, then a re-review of the range of the fix. A mutation proof is repeated only when the fix changed its production line.
|
|
121
121
|
|
|
122
|
-
A red gate on the commit of a fixer is a critical finding with source `gate`. Its file is the gate record, and the record holds the output. No review runs on that tip, and the next round starts. The re-review after that round covers each fix since the last review and repeats each proof that one of them made due. A red gate after round 4 blocks the ticket, as does a red gate on the commit of the implementer or of a
|
|
122
|
+
A red gate on the commit of a fixer is a critical finding with source `gate`. Its file is the gate record, and the record holds the output. No review runs on that tip, and the next round starts. The re-review after that round covers each fix since the last review and repeats each proof that one of them made due. A red gate after round 4 blocks the ticket, as does a red gate on the commit of the implementer or of a test-writer correction.
|
|
123
123
|
|
|
124
124
|
- Rounds 1 and 2 run on the first entry of the implementer list.
|
|
125
125
|
- Rounds 3 and 4 run on the second entry of the implementer list. When the list has one entry, they stay on it and the ledger says so.
|
|
@@ -172,7 +172,7 @@ Every report has `status` (`DONE`, `DONE_WITH_CONCERNS`, `NEEDS_CONTEXT`, `BLOCK
|
|
|
172
172
|
|
|
173
173
|
| Role | Template | More fields | Checks after the schema |
|
|
174
174
|
|---|---|---|---|
|
|
175
|
-
| test-writer | `prompts/test-writer.md` | `locked` (paths), `red` (`test`, `line`) | With DONE: one commit, one locked file and one red line or more. Each locked path is in a commit that the role made. Each `line` is in the log. A locked file of a merged ticket that the role changed is in `locked`. |
|
|
175
|
+
| test-writer | `prompts/test-writer.md` | `locked` (test file paths), `red` (`test`, `line`) | The driver reads a test node id `<file>::<test>` in `locked` as its file. With DONE: one commit, one locked file and one red line or more. Each locked path is in a commit that the role made. Each `line` is in the log. A locked file of a merged ticket that the role changed is in `locked`. |
|
|
176
176
|
| implementer | `prompts/implementer.md` | `green` | With DONE: one commit or more. |
|
|
177
177
|
| reviewer | `prompts/reviewer.md` | `findings` (`severity`, `file`, `line`, `title`, `evidence`, `kind`), `mutations` (`test`, `file`, `line`, `red`); `commits` is empty | In the first review of a tip: `mutations` is required, with a proof for each locked test file (`test` is `<locked file>::<test name>`). Each `red` is in the log. |
|
|
178
178
|
| correctness | `prompts/correctness.md` | the same as the reviewer; `commits` is empty | `mutations` is optional. |
|
|
@@ -190,7 +190,7 @@ A question is a bounded choice for the operator. `driver_decisions.py` `KINDS` h
|
|
|
190
190
|
|
|
191
191
|
| Kind | Question id | Asked when | Run exit | Options |
|
|
192
192
|
|---|---|---|---|---|
|
|
193
|
-
| `locked-test` | `<NN>-locked-test-<n>` | An implementer reports BLOCKED with a concern that starts with `locked test contradicts spec:`. | 2 | `overrule-test`: the test-writer corrects the test, then the implementer runs again. `overrule-implementer` (recommended): the implementer runs again. |
|
|
193
|
+
| `locked-test` | `<NN>-locked-test-<n>` | An implementer or a fixer reports BLOCKED with a concern that starts with `locked test contradicts spec:`. | 2 | `overrule-test`: the test-writer corrects the test from that report, then the implementer runs again, and the fix rounds count from 0: the corrected tests are a new contract. `overrule-implementer` (recommended): the implementer runs again. |
|
|
194
194
|
| `finding` | `<NN>-finding-<n>` | A finding is open after fix round 4. One question for each finding. | 1 | `accept`: close the finding; the ticket merges when each of its findings is accepted. `split` (recommended) and `redesign`: the ticket stays BLOCKED, with the ruling in the ledger. |
|
|
195
195
|
| `final-finding` | `final-finding-<n>` | A finding is open after the second round of the final review. | 2 | `fix` (recommended): the finding stays open and nothing is pushed. `accept`: close the finding. |
|
|
196
196
|
|
|
@@ -69,13 +69,20 @@ def _check_out_ticket(ticket: Ticket, slot_number: int, entry: dict, run: Run) -
|
|
|
69
69
|
return driver_slots.head(slot)
|
|
70
70
|
|
|
71
71
|
|
|
72
|
+
def _blocked_report(number: str, run: Run) -> str:
|
|
73
|
+
"""The report of the last `locked-test` stop of the ticket: an implementer's or a fixer's."""
|
|
74
|
+
asked = [q["evidence"][0] for q in run.state.read()["run"]["questions"]
|
|
75
|
+
if (q["ticket"], q["kind"]) == (number, "locked-test")]
|
|
76
|
+
return asked[-1] if asked else str(OUT / number / "implementer-0.json")
|
|
77
|
+
|
|
78
|
+
|
|
72
79
|
def _prompt_values(ticket: Ticket, slot: str, run: Run) -> dict[str, str]:
|
|
73
80
|
"""The pointers every prompt of the ticket gets. An agent works in its slot, so paths are relative to it."""
|
|
74
81
|
folder = OUT / ticket.number
|
|
75
82
|
return {"spec": os.path.relpath(run.args.spec, slot), "seam": ticket.seam, "pr_branch": run.pr,
|
|
76
83
|
"branch": f"ticket/{ticket.number}", "ticket": os.path.relpath(Path(run.args.tickets) / ticket.file, slot),
|
|
77
84
|
"correctness": str(driver_review.CORRECTNESS),
|
|
78
|
-
"blocked_report": os.path.relpath(
|
|
85
|
+
"blocked_report": os.path.relpath(_blocked_report(ticket.number, run), slot),
|
|
79
86
|
"writer_report": os.path.relpath(folder / "test-writer-0.json", slot)}
|
|
80
87
|
|
|
81
88
|
|
|
@@ -37,7 +37,7 @@ KINDS = {
|
|
|
37
37
|
{"overrule-test": Option(
|
|
38
38
|
"The test is wrong. The test-writer corrects it in one correction run, then the implementer runs again.",
|
|
39
39
|
"Ruling: {ticket} locked test overruled: the test-writer corrects {subject}, then the implementer runs "
|
|
40
|
-
"again.", {"phase": "test-writer", "note": "overruled"}),
|
|
40
|
+
"again.", {"phase": "test-writer", "note": "overruled", "round": 0}), # new tests, new fix rounds
|
|
41
41
|
"overrule-implementer": Option(
|
|
42
42
|
"The test is right. The implementer runs again and makes the test green as it is.",
|
|
43
43
|
"Ruling: {ticket} locked test stands: the implementer is overruled on {subject} and runs again.",
|
|
@@ -73,8 +73,9 @@ KINDS = {
|
|
|
73
73
|
|
|
74
74
|
|
|
75
75
|
def contradicted(role: str, report: dict) -> str:
|
|
76
|
-
"""The locked tests
|
|
77
|
-
|
|
76
|
+
"""The locked tests that a BLOCKED report of an implementer or a fixer says contradict the spec; "" when it is not
|
|
77
|
+
that stop."""
|
|
78
|
+
if role not in ("implementer", "fixer") or report.get("status") != "BLOCKED":
|
|
78
79
|
return ""
|
|
79
80
|
return "; ".join(filter(None, (concern[len(CONTRADICTS):].strip() for concern in report["concerns"]
|
|
80
81
|
if concern.startswith(CONTRADICTS))))
|
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
"""A gated ticket tip in, a reviewed tip out: the review, the
|
|
1
|
+
"""A gated ticket tip in, a reviewed tip out: the review, the test-writer corrections and the fix rounds.
|
|
2
2
|
|
|
3
3
|
Reports of a ticket are `<role>-<K>.json`: K is the fix round the agent follows (reviewer, correctness) or is
|
|
4
|
-
(fixer). A
|
|
4
|
+
(fixer). A test-writer correction after the review of round K is `test-writer-<K+1>.json`.
|
|
5
5
|
"""
|
|
6
6
|
from __future__ import annotations
|
|
7
7
|
|
|
@@ -31,7 +31,7 @@ class Job:
|
|
|
31
31
|
previous: str = "" # the tip before the last fix; "" makes the next review a full one
|
|
32
32
|
proofs: set[tuple[str, int]] = field(default_factory=set) # mutation proofs so far, at `tip`
|
|
33
33
|
again: set[tuple[str, int]] = field(default_factory=set) # each proof at `tip` that a fix made due again
|
|
34
|
-
corrected: bool = False # the last step was a
|
|
34
|
+
corrected: bool = False # the last step was a test-writer correction
|
|
35
35
|
red: bool = False # the gate of `tip` is red, so no review saw it
|
|
36
36
|
|
|
37
37
|
def pointers(self, review: int) -> dict[str, str]:
|
|
@@ -99,21 +99,21 @@ def _review(job: Job) -> list[dict]:
|
|
|
99
99
|
return found
|
|
100
100
|
|
|
101
101
|
|
|
102
|
-
def
|
|
103
|
-
"""One test-writer correction for the
|
|
102
|
+
def _correct(job: Job, findings: list[dict]) -> None:
|
|
103
|
+
"""One test-writer correction for the findings on the locked tests. It is outside the fix rounds."""
|
|
104
104
|
run, number = job.run, job.ticket.number
|
|
105
105
|
if job.corrected:
|
|
106
|
-
titles = "; ".join(finding["title"] for finding in
|
|
107
|
-
raise Blocked(f"the review: a
|
|
106
|
+
titles = "; ".join(finding["title"] for finding in findings)
|
|
107
|
+
raise Blocked(f"the review: a locked-test finding is open after the test-writer correction: {titles}")
|
|
108
108
|
run.restore_locked_files(number, job.slot)
|
|
109
|
-
with during("the
|
|
109
|
+
with during("the test-writer correction"):
|
|
110
110
|
report = run.agent("test-writer", number, job.slot, job.pointers(job.round), notes=["correction"],
|
|
111
111
|
fix_round=job.round + 1)
|
|
112
112
|
job.tip, job.previous, job.corrected = driver_slots.head(job.slot), "", True
|
|
113
113
|
entry = run.state.read()["tickets"][number]
|
|
114
114
|
run.state.ticket(number, tip=job.tip, **driver_review.written(job.slot, entry, report["locked"], job.tip))
|
|
115
|
-
for finding in
|
|
116
|
-
ledger(f"Ticket {number}:
|
|
115
|
+
for finding in findings:
|
|
116
|
+
ledger(f"Ticket {number}: {finding['kind']} correction in {finding['file']}, outside the fix rounds: "
|
|
117
117
|
f"{finding['title']}: {finding['evidence']}. The test-writer corrected it at {job.tip[:7]}.")
|
|
118
118
|
|
|
119
119
|
|
|
@@ -137,18 +137,19 @@ def _fix(job: Job) -> None:
|
|
|
137
137
|
# No review saw `before`: the next review covers the fix before it too, and its due proofs stay due.
|
|
138
138
|
due |= set(driver_review.follow(job.slot, before, job.tip, job.again))
|
|
139
139
|
else:
|
|
140
|
-
# No review has seen a
|
|
140
|
+
# No review has seen a test-writer correction, so the review after a fix on a corrected tip is a full one.
|
|
141
141
|
job.previous = "" if job.corrected else before
|
|
142
142
|
job.corrected, job.proofs, job.again = False, set(followed), due
|
|
143
143
|
|
|
144
144
|
|
|
145
145
|
def _act_on(job: Job, found: list[dict]) -> bool:
|
|
146
|
-
"""Do what the findings of a review ask for: a
|
|
147
|
-
|
|
146
|
+
"""Do what the findings of a review ask for: a correction, a fix round, or a stop. True: none is open."""
|
|
147
|
+
locked = job.run.state.read()["tickets"][job.ticket.number]["locked"]
|
|
148
|
+
to_correct, to_fix = driver_review.route(found, locked)
|
|
148
149
|
# A changed locked file is fixed first: a correction on top of it would make the changed file the reference.
|
|
149
150
|
locked_file_changed = any(finding["source"] == "driver" for finding in found)
|
|
150
|
-
if
|
|
151
|
-
|
|
151
|
+
if to_correct and not locked_file_changed:
|
|
152
|
+
_correct(job, to_correct)
|
|
152
153
|
if not to_fix:
|
|
153
154
|
return False
|
|
154
155
|
if not to_fix:
|
|
@@ -100,6 +100,11 @@ def _locked_errors(report: dict, context: dict) -> list[str]:
|
|
|
100
100
|
if not any(test.startswith(path) for test in tests)]
|
|
101
101
|
|
|
102
102
|
|
|
103
|
+
def locked_files(listed: list[str]) -> list[str]:
|
|
104
|
+
"""The files of a `locked` list: a test node id `<file>::<test>` names its file. In order, each file once."""
|
|
105
|
+
return list(dict.fromkeys(entry.partition("::")[0] for entry in listed))
|
|
106
|
+
|
|
107
|
+
|
|
103
108
|
def validate(role: str, path: str, context: dict) -> list[str]:
|
|
104
109
|
try:
|
|
105
110
|
report = json.loads(Path(path).read_text())
|
|
@@ -111,5 +116,7 @@ def validate(role: str, path: str, context: dict) -> list[str]:
|
|
|
111
116
|
return [f"{'.'.join(map(str, e.path)) or 'report'}: {e.message}" for e in found]
|
|
112
117
|
# The schema types only the role's own fields: any other field of the report is dropped.
|
|
113
118
|
report = {name: value for name, value in report.items() if name in rules["properties"]}
|
|
119
|
+
if "locked" in report:
|
|
120
|
+
report["locked"] = locked_files(report["locked"])
|
|
114
121
|
return (_commit_errors(report, context) + _owner_errors(report, context) + _correction_errors(report, context)
|
|
115
122
|
+ _merged_errors(report, context) + _proof_errors(report, context) + _locked_errors(report, context))
|
|
@@ -76,10 +76,13 @@ def written(slot: str, entry: dict, listed: list[str], tip: str) -> dict:
|
|
|
76
76
|
return {"test_writer_sha": tip, "locked": list(dict.fromkeys(entry["locked"] + listed)), "held": held}
|
|
77
77
|
|
|
78
78
|
|
|
79
|
-
def route(findings: list[dict]) -> tuple[list[dict], list[dict]]:
|
|
80
|
-
"""(the
|
|
81
|
-
|
|
82
|
-
|
|
79
|
+
def route(findings: list[dict], locked: list[str]) -> tuple[list[dict], list[dict]]:
|
|
80
|
+
"""(the findings for a test-writer correction; the other findings that are severe enough for a fix round).
|
|
81
|
+
A correction gets each fixture finding, whatever its severity, and each severe test finding on a `locked` file:
|
|
82
|
+
the fixer may not edit a locked file, so a fix round cannot close it."""
|
|
83
|
+
correct = [finding for finding in findings if finding["kind"] == "fixture"
|
|
84
|
+
or (finding["kind"] == "test" and finding["file"] in locked and finding["severity"] in OPEN)]
|
|
85
|
+
return correct, [finding for finding in findings if finding not in correct and finding["severity"] in OPEN]
|
|
83
86
|
|
|
84
87
|
|
|
85
88
|
def _hunks(slot: str, old: str, new: str, file: str) -> list[tuple[int, int, int, int]]:
|
|
@@ -100,6 +100,8 @@ class Run:
|
|
|
100
100
|
merged = driver_merge.protected(self.state.read()["tickets"], number)
|
|
101
101
|
options["check"] = {**(options.get("check") or {}), "merged": merged, "pr": self.tip()}
|
|
102
102
|
report = _attempts(self, role, number, slot, values, **options)
|
|
103
|
+
if role == "test-writer":
|
|
104
|
+
report["locked"] = driver_report.locked_files(report["locked"]) # as validate read it
|
|
103
105
|
if subject := driver_decisions.contradicted(role, report):
|
|
104
106
|
# The implementer's commits stay: after the ruling the ticket goes on from them.
|
|
105
107
|
evidence = OUT / number / f"{role}-{options.get('fix_round', 0)}.json"
|
|
@@ -1,4 +1,6 @@
|
|
|
1
1
|
|
|
2
|
-
This call is a
|
|
3
|
-
|
|
4
|
-
|
|
2
|
+
This call is a correction of the locked tests, not a new test set. Read in {findings} each finding of kind `fixture` and each critical or important finding of kind `test` on a locked file.
|
|
3
|
+
A `fixture` finding: a locked test encodes a stale path, format, fixture layout or setup detail. Correct only that detail and keep what each test asserts.
|
|
4
|
+
A `test` finding: a locked test is too weak, for example it stays green when the production line it protects is broken. Make the test assert the behavior of its criterion so that it fails on that break. Do not remove an assertion.
|
|
5
|
+
For this you may read the implementation.
|
|
6
|
+
The implementation exists, so a corrected test is green: for `red`, break the production line that the test protects, run the test, print its failing output, then revert the edit. In `locked`, list each test file you corrected: test file paths only, not test node ids. Unlike `mutations[].test` of the reviewer, `locked` takes no `::<test name>`.
|
|
@@ -12,7 +12,7 @@ The output section of the checks file does not apply here: your findings go into
|
|
|
12
12
|
Read file content from git at {start_sha} (`git diff`, `git show {start_sha}:<path>`), never from the working tree, and run no tests: another agent makes temporary edits in this worktree while you work.
|
|
13
13
|
The one exception is the input of the driver, which git can lack because `.agent/` is ignored: read the spec, the ticket and the findings file from the working tree. The other agent does not edit them.
|
|
14
14
|
Report only a defect that you can show with a concrete sequence of events. `mutations` is optional for this pass.
|
|
15
|
-
Give each finding a severity (critical, important or minor), the file and line, a title, the evidence, and a kind: `
|
|
15
|
+
Give each finding a severity (critical, important or minor), the file and line, a title, the evidence, and a kind: `fixture` when a locked test encodes a stale path, format or setup detail; `test` when a test is weak or missing, as a locked test that stays green when you break the line it protects; `code` for anything else. The test-writer corrects each `fixture` finding and each critical or important `test` finding on a locked file, and the fixer gets the rest.
|
|
16
16
|
Do not commit and do not dispatch subagents. Leave the tree as you found it.
|
|
17
17
|
|
|
18
18
|
Write one JSON report at {report}. When your sandbox is read-only, give the JSON as your whole final message and the driver saves it there. The report matches this schema:
|
|
@@ -7,8 +7,9 @@ Branch: {branch}, at commit {start_sha}
|
|
|
7
7
|
Locked files: {locked}
|
|
8
8
|
Findings of the review: {findings}
|
|
9
9
|
|
|
10
|
-
Fix each finding of severity critical or important in the findings file. Leave the minor findings: the final review gets them. Leave each finding of kind `fixture
|
|
10
|
+
Fix each finding of severity critical or important in the findings file. Leave the minor findings: the final review gets them. Leave each finding of kind `fixture` and each finding of kind `test` on a locked file: the test-writer corrects those.
|
|
11
11
|
Never edit a locked file. The one exception is a finding `locked file changed`: make that file equal to its content at {test_writer_sha}.
|
|
12
|
+
If a locked test contradicts the spec or the ticket, so that no fix can make it green, stop and report BLOCKED with a concern that starts with `locked test contradicts spec:`, then the test and the evidence.
|
|
12
13
|
Use targeted tests while you work. The driver runs the gate on your final commit, and a re-review reads the lines you changed.
|
|
13
14
|
A finding from the `gate` is a red gate: its file is the gate record. Read the output there and make the gate green.
|
|
14
15
|
Commit your work on the checked-out ticket branch. Do not dispatch subagents.
|
|
@@ -7,7 +7,7 @@ Start commit, which holds the locked tests when the ticket has them: {start_sha}
|
|
|
7
7
|
Locked files: {locked}
|
|
8
8
|
|
|
9
9
|
Make the locked tests green and meet each criterion of the ticket. For internal code, work in vertical test-first slices: one failing test, the smallest code that passes it, then the next slice.
|
|
10
|
-
Never edit a locked file. If a locked test contradicts the spec or the ticket, stop and report BLOCKED with the
|
|
10
|
+
Never edit a locked file. If a locked test contradicts the spec or the ticket, stop and report BLOCKED with a concern that starts with `locked test contradicts spec:`, then the test and the evidence.
|
|
11
11
|
Commit your work on the checked-out ticket branch. Do not dispatch subagents.
|
|
12
12
|
Run targeted tests while you work. The driver runs the gate on your final commit.
|
|
13
13
|
In `green`, give the summary lines of the passing test output.
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
|
|
2
|
-
This call is a correction after a ruling, not a new test set. The implementer reported that a locked test contradicts the spec or the ticket, and the operator ruled that the test is wrong: read `concerns` in {blocked_report}.
|
|
2
|
+
This call is a correction after a ruling, not a new test set. The implementer or the fixer reported that a locked test contradicts the spec or the ticket, and the operator ruled that the test is wrong: read `concerns` in {blocked_report}.
|
|
3
3
|
Correct only the tests that those concerns name, so that they state what the spec and the ticket require. Keep each other test as it is.
|
|
4
4
|
The branch can hold commits of the implementer, and the tree can hold its uncommitted work: change neither, and commit only the test files you correct.
|
|
5
5
|
A corrected test fails because the behavior is missing: run it and print that failing output. In `locked`, list each test file you corrected.
|
|
@@ -11,7 +11,7 @@ Check the diff against the ticket and the spec, then for quality. Check that eac
|
|
|
11
11
|
In round 1, for each locked test, break the production line the test protects, run the test, print its failing output, then revert the edit. A test that stays green is a critical finding. Run again each mutation that the test-writer report names in `concerns`.
|
|
12
12
|
A mutation proof is keyed to its test and its production line: a later round repeats it only when that line changed.
|
|
13
13
|
In `mutations`, give for each proof the test as `<locked file>::<test name>`, the production file and line, and one failure line copied exactly from the output you printed.
|
|
14
|
-
Give each finding a severity (critical, important or minor), the file and line, a title, the evidence, and a kind: `
|
|
14
|
+
Give each finding a severity (critical, important or minor), the file and line, a title, the evidence, and a kind: `fixture` when a locked test encodes a stale path, format or setup detail; `test` when a test is weak or missing, as a locked test that stays green when you break the line it protects; `code` for anything else. The test-writer corrects each `fixture` finding and each critical or important `test` finding on a locked file, and the fixer gets the rest.
|
|
15
15
|
Do not commit and do not dispatch subagents. Leave the tree as you found it.
|
|
16
16
|
|
|
17
17
|
Write one JSON report at {report}. When your sandbox is read-only, give the JSON as your whole final message and the driver saves it there. The report matches this schema:
|
|
@@ -10,7 +10,7 @@ Write one black-box acceptance test for each criterion of the ticket, through th
|
|
|
10
10
|
Run each test and confirm that it fails because the behavior is missing, not because of an import error or a syntax error. Print that failing output.
|
|
11
11
|
A test that is green before any implementation is not a proof. Rewrite it until it fails, or name in `concerns` the production-code mutation that turns it red.
|
|
12
12
|
Commit the tests red on the checked-out branch. Do not write implementation code and do not dispatch subagents.
|
|
13
|
-
In `locked`, list each test file of your commit. In `red`, give for each test one failure line, copied exactly from the output you printed.
|
|
13
|
+
In `locked`, list each test file of your commit: test file paths only, not test node ids (`tests/x.py`, not `tests/x.py::test_a`). In `red`, give for each test one failure line, copied exactly from the output you printed.
|
|
14
14
|
|
|
15
15
|
Write one JSON report at {report} that matches this schema:
|
|
16
16
|
|
|
@@ -353,10 +353,11 @@ def test_a_template_of_another_shape_is_an_error_and_every_text_of_a_question_is
|
|
|
353
353
|
|
|
354
354
|
# ---------------------------------------------------------------- questions
|
|
355
355
|
|
|
356
|
-
def
|
|
356
|
+
def test_only_an_implementer_or_a_fixer_blocked_on_the_contradiction_concern_is_a_stop():
|
|
357
357
|
report = {"status": "BLOCKED", "concerns": [CONTRA, "other", "locked test contradicts spec: b::c", driver_decisions.CONTRADICTS]}
|
|
358
358
|
assert driver_decisions.contradicted("implementer", report) == "tests/test_01.py::test_x; b::c"
|
|
359
|
-
assert driver_decisions.contradicted("fixer", report) == ""
|
|
359
|
+
assert driver_decisions.contradicted("fixer", report) == "tests/test_01.py::test_x; b::c"
|
|
360
|
+
assert driver_decisions.contradicted("reviewer", report) == ""
|
|
360
361
|
assert driver_decisions.contradicted("implementer", {**report, "status": "DONE"}) == ""
|
|
361
362
|
assert driver_decisions.contradicted("implementer", {}) == "", "a rejected report is no report"
|
|
362
363
|
assert driver_decisions.contradicted("implementer", {"status": "BLOCKED", "concerns": [driver_decisions.CONTRADICTS]}) == ""
|
|
@@ -127,7 +127,7 @@ def test_a_changed_locked_file_is_fixed_before_a_fixture_finding_is_corrected(tm
|
|
|
127
127
|
assert (entry["round"], entry["test_writer_sha"]) == (1, rig.calls("01", "test-writer")[1]["sha"])
|
|
128
128
|
assert entry["locked"] == ["tests/test_01.py"] and (rig.folder() / "test-writer-2.json").exists()
|
|
129
129
|
assert rig.led("fixture correction in tests/test_01.py, outside the fix rounds: STALE: STALE.")
|
|
130
|
-
assert "This call is a
|
|
130
|
+
assert "This call is a correction of the locked tests" in rig.prompt("test-writer-2") and "findings-1.json" in rig.prompt("test-writer-2")
|
|
131
131
|
|
|
132
132
|
|
|
133
133
|
def test_a_locked_file_check_that_git_cannot_make_is_a_finding(tmp_path):
|
|
@@ -141,7 +141,7 @@ def test_a_fixture_finding_that_comes_back_after_its_correction_blocks_the_ticke
|
|
|
141
141
|
stale = finding("STALE", kind="fixture", file="tests/test_01.py")
|
|
142
142
|
correction = {**tw("01"), "files": {"tests/test_01.py": "# corrected\n"}, "commit": "test: 01 fixture fix"}
|
|
143
143
|
rig = Inside(tmp_path).run(plan(**{"01/test-writer": [tw("01"), correction], "01/reviewer": [review("01", [stale])] * 2}))
|
|
144
|
-
assert blocked(rig).endswith("BLOCKED at the review: a
|
|
144
|
+
assert blocked(rig).endswith("BLOCKED at the review: a locked-test finding is open after the test-writer correction: STALE.")
|
|
145
145
|
assert len(rig.calls("01", "test-writer")) == 2 and rig.state()["tickets"]["01"]["round"] == 0
|
|
146
146
|
|
|
147
147
|
|
|
@@ -160,10 +160,78 @@ def test_an_open_finding_survives_the_fixture_correction_of_its_review_and_start
|
|
|
160
160
|
assert [gate["commit"] for gate in rig.walk_gates()] == [rig.sha("01", "implementer"), rig.sha("01", "fixer")]
|
|
161
161
|
|
|
162
162
|
|
|
163
|
+
def test_a_test_finding_on_a_locked_file_gets_the_test_writer_correction_not_a_fix_round(tmp_path):
|
|
164
|
+
# A failed mutation proof: the fixer may not edit the locked file, so a fix round could never close it.
|
|
165
|
+
vacuous = finding("VACUOUS", "critical", kind="test", file="tests/test_01.py")
|
|
166
|
+
correction = {**tw("01"), "files": {"tests/test_01.py": "# stronger\n"}, "commit": "test: 01 stronger test"}
|
|
167
|
+
rig = Inside(tmp_path).run(plan(**{"01/test-writer": [tw("01"), correction],
|
|
168
|
+
"01/reviewer": [review("01", [vacuous]), review("01")]}))
|
|
169
|
+
assert rig.code == 0, rig.ledger()
|
|
170
|
+
order = [call["role"] for call in rig.calls("01")]
|
|
171
|
+
assert order == ["test-writer", "implementer", "reviewer", "test-writer", "reviewer"]
|
|
172
|
+
entry = rig.state()["tickets"]["01"]
|
|
173
|
+
assert (entry["round"], entry["test_writer_sha"]) == (0, rig.calls("01", "test-writer")[1]["sha"])
|
|
174
|
+
assert rig.led("correction in tests/test_01.py, outside the fix rounds: VACUOUS: VACUOUS.")
|
|
175
|
+
# Only a severe finding of kind `test` on a locked file goes to the test-writer.
|
|
176
|
+
minor, elsewhere = finding("M", "minor", kind="test", file="tests/test_01.py"), finding("E", kind="test")
|
|
177
|
+
assert driver_review.route([vacuous, minor, elsewhere], ["tests/test_01.py"]) == ([vacuous], [elsewhere])
|
|
178
|
+
|
|
179
|
+
|
|
180
|
+
def test_a_test_writer_that_lists_test_node_ids_in_locked_locks_their_files(tmp_path):
|
|
181
|
+
# Issue #71: a node id `<file>::<test>` in `locked` names its file, for the first run and for a correction.
|
|
182
|
+
vacuous = finding("VACUOUS", "critical", kind="test", file="tests/test_01.py")
|
|
183
|
+
ids = ["tests/test_01.py::test_x", "tests/test_01.py::test_y"]
|
|
184
|
+
first = {**tw("01"), "report": {**tw("01")["report"], "locked": ids}}
|
|
185
|
+
correction = {**first, "files": {"tests/test_01.py": "# stronger\n"}, "commit": "test: 01 stronger test"}
|
|
186
|
+
rig = Inside(tmp_path).run(plan(**{"01/test-writer": [first, correction],
|
|
187
|
+
"01/reviewer": [review("01", [vacuous]), review("01")]}))
|
|
188
|
+
assert rig.code == 0, rig.ledger()
|
|
189
|
+
assert [call["role"] for call in rig.calls("01")].count("test-writer") == 2 # no rerun
|
|
190
|
+
assert rig.state()["tickets"]["01"]["locked"] == ["tests/test_01.py"]
|
|
191
|
+
|
|
192
|
+
|
|
193
|
+
CONTRA = "locked test contradicts spec: tests/test_01.py::test_x"
|
|
194
|
+
OVERRULED = {**tw("01"), "files": {"tests/test_01.py": "def test_x():\n assert '01' == '01'\n"},
|
|
195
|
+
"commit": "test: 01 overruled"}
|
|
196
|
+
AGAIN = impl("01", text="def fn_01():\n return 'again'\n")
|
|
197
|
+
|
|
198
|
+
|
|
199
|
+
def fixer_stops():
|
|
200
|
+
return {"report": {"status": "BLOCKED", "commits": [], "files": [], "concerns": [CONTRA], "green": [], "ticket": "01"}}
|
|
201
|
+
|
|
202
|
+
|
|
203
|
+
def test_a_fixer_that_reports_a_wrong_locked_test_opens_the_locked_test_question(tmp_path):
|
|
204
|
+
steps = plan(**{"01/reviewer": [review("01", [STUCK]), review("01")], "01/fixer": [fixer_stops()],
|
|
205
|
+
"01/test-writer": [tw("01"), OVERRULED], "01/implementer": [impl("01"), AGAIN]})
|
|
206
|
+
rig = Inside(tmp_path).run(steps)
|
|
207
|
+
assert rig.code == 2, rig.ledger()
|
|
208
|
+
(asked,) = rig.state()["run"]["questions"]
|
|
209
|
+
assert (asked["id"], asked["evidence"][0]) == ("01-locked-test-1", ".agent/implement/01/fixer-1.json")
|
|
210
|
+
rig.run(steps, "--answer", "01-locked-test-1=overrule-test")
|
|
211
|
+
assert rig.code == 0, rig.ledger()
|
|
212
|
+
order = [call["role"] for call in rig.calls("01")]
|
|
213
|
+
assert order == ["test-writer", "implementer", "reviewer", "fixer", "test-writer", "implementer", "reviewer"]
|
|
214
|
+
# The correction reads the concerns of the fixer that stopped, not those of the implementer.
|
|
215
|
+
assert "01/fixer-1.json" in rig.prompt("test-writer-0") and "implementer-0.json" not in rig.prompt("test-writer-0")
|
|
216
|
+
|
|
217
|
+
|
|
218
|
+
def test_the_fix_rounds_count_from_0_after_an_overruled_test(tmp_path):
|
|
219
|
+
stuck = review("01", [STUCK])
|
|
220
|
+
steps = plan(**{"01/reviewer": [stuck] * 5 + [review("01")],
|
|
221
|
+
"01/fixer": [fix("01", 1), fix("01", 2), fix("01", 3), fixer_stops(), fix("01", 5)],
|
|
222
|
+
"01/test-writer": [tw("01"), OVERRULED], "01/implementer": [impl("01"), AGAIN]})
|
|
223
|
+
rig = Inside(tmp_path).run(steps)
|
|
224
|
+
assert rig.code == 2 and rig.state()["tickets"]["01"]["round"] == 4, rig.ledger()
|
|
225
|
+
rig.run(steps, "--answer", "01-locked-test-1=overrule-test")
|
|
226
|
+
# The corrected tests are a new contract: the first finding on them gets a fix round, not the round-4 stop.
|
|
227
|
+
assert rig.code == 0, rig.ledger()
|
|
228
|
+
assert rig.state()["tickets"]["01"]["round"] == 1 and not rig.led("after round 4")
|
|
229
|
+
|
|
230
|
+
|
|
163
231
|
def test_a_fixture_correction_that_is_rejected_blocks_the_ticket(tmp_path):
|
|
164
232
|
stale = finding("STALE", kind="fixture", file="tests/test_01.py")
|
|
165
233
|
rig = Inside(tmp_path).run(plan(**{"01/test-writer": [tw("01"), {}, {}], "01/reviewer": [review("01", [stale])]}))
|
|
166
|
-
assert "BLOCKED at the
|
|
234
|
+
assert "BLOCKED at the test-writer correction: report .agent/implement/01/test-writer-1.json is not" in blocked(rig)
|
|
167
235
|
|
|
168
236
|
|
|
169
237
|
# ---------------------------------------------------------------- the locked files of the merged tickets
|
package/skills-manifest.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"schemaVersion": 1,
|
|
3
|
-
"generatedAt": "2026-10-
|
|
3
|
+
"generatedAt": "2026-10-15T21:59:51.763Z",
|
|
4
4
|
"skills": [
|
|
5
5
|
{
|
|
6
6
|
"name": "backlog",
|
|
@@ -46,8 +46,8 @@
|
|
|
46
46
|
},
|
|
47
47
|
{
|
|
48
48
|
"name": "implement-spec",
|
|
49
|
-
"version": "0.1.
|
|
50
|
-
"hash": "sha256-
|
|
49
|
+
"version": "0.1.14",
|
|
50
|
+
"hash": "sha256-79c7cae6498225802c8d12a8e5c58c74ec0f2ef3b8889bc1e529b3115407e3e9",
|
|
51
51
|
"description": "Implement a specification in code."
|
|
52
52
|
},
|
|
53
53
|
{
|