@jesdi/skills 0.0.43 → 0.0.45

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@jesdi/skills",
3
- "version": "0.0.43",
3
+ "version": "0.0.45",
4
4
  "description": "jesdi's agent skills (content package consumed by @jesdi/skills-cli)",
5
5
  "license": "MIT",
6
6
  "repository": {
@@ -21,7 +21,9 @@ A comment in the code is for a line that looks wrong or removable without it.
21
21
  | `driver_merge.py` | Merges by git plumbing, the locked-file check of a landing, the move of the PR ref |
22
22
  | `driver_gate.py` | One run of the gate command and its record |
23
23
  | `driver_slots.py` | The worktrees that tickets run in, and the `git` helper |
24
- | `driver_state.py` | `state.json` (the resume contract) and the lock of one driver per repository |
24
+ | `driver_state.py` | `state.json` (the resume contract) and the kernel lock |
25
+ | `driver_names.py` | The names of one run: its ticket branches, its recovery ref and its lock file |
26
+ | `driver_legacy.py` | The take-over of a run that an older driver started; delete it when no such run is left |
25
27
  | `driver_decisions.py`, `driver_answers.py`, `driver_review_page.py` | Operator questions, their answers, the page that shows them |
26
28
  | `driver_tidy.py`, `driver_summary.py` | The history with one commit per ticket; `summary.md` |
27
29
 
@@ -68,8 +70,15 @@ and do not know about the run, except `driver_run.py`, which the three deciding
68
70
  is not clean, so a restore left in the tree would turn the next gate red and use a fix round. The correction's
69
71
  prompt names the restored files (`prompts/restored.md`): this is the one path for a change of an earlier ticket's
70
72
  locked test that a later ticket's interface needs.
71
- - The locked files of merged tickets are checked when a ticket lands (`driver_merge.refused`): the tree that lands
72
- must have each of them as the PR tip has it. A ticket's test-writer may change one only when it lists it.
73
+ - The driver, not the correction, decides whether a restored file fails with the ticket's code: in the fix loop it
74
+ gates the restore commit, whose parent has a green gate. A red gate goes to the correction as evidence
75
+ (`prompts/restored-red.md`), and a DONE correction must then list a restored file. An agent's judgement of
76
+ "this test does not fail" missed a red file and led to a refused merge (issue #90). After `overrule-test` the
77
+ parent has no green gate, so a red gate proves nothing there: the next review catches the file instead.
78
+ - The locked files of merged tickets are checked before each review (`driver_review.merged_changes`): a change is a
79
+ `fixture` finding, so it goes to a correction, where the restore and its gate decide. They are checked again when
80
+ a ticket lands (`driver_merge.refused`): the tree that lands must have each of them as the PR tip has it. A
81
+ ticket's test-writer may change one only when it lists it.
73
82
  - A conflict resolution, of a merge or of the rebase, may not change a locked file.
74
83
 
75
84
  ### The PR branch
@@ -82,7 +91,7 @@ and do not know about the run, except `driver_run.py`, which the three deciding
82
91
  - The gate runs only on a commit the driver is about to trust, and one green record settles that commit.
83
92
  A gate on a dirty tree, or one that leaves the tree changed or HEAD moved, is red.
84
93
  - The end of run rewrites the branch one time, before the first final review, and never after a review. A history
85
- that origin has is never rewritten. `refs/implement/recovery` holds the tip before the rewrite.
94
+ that origin has is never rewritten. `refs/implement-spec/<key>/recovery` holds the tip before the rewrite.
86
95
  - A rewrite never changes a file: the driver compares trees (`driver_tidy.same_tree`) before it moves the branch.
87
96
  - The push sends the reviewed commit, not the branch name, with no force.
88
97
 
@@ -103,8 +112,11 @@ and do not know about the run, except `driver_run.py`, which the three deciding
103
112
  `finish_interrupted_move` completes it at the next start.
104
113
  - At a resume, git must confirm each recorded tip (`driver_state.verified`). A ticket that git contradicts goes
105
114
  back to `implement` from its test-writer commit.
106
- - The lock of one driver per repository is a kernel lock on a file in the git common directory. The kernel frees
107
- it when the process ends, so it is never stale.
115
+ - The lock of one driver per PR branch is a kernel lock on a file in the git common directory. The kernel frees
116
+ it when the process ends, so it is never stale. Every ref that a run writes has the PR branch in its name
117
+ as one escaped name component. `driver_names.Names` (`Run.names`) makes every name of a run, so runs on
118
+ different PR branches share no ref. `driver_legacy.resume` takes over a run that an older driver started, once,
119
+ after the lock and before any check reads a ref.
108
120
 
109
121
  ### Agents
110
122
 
@@ -23,7 +23,9 @@ Before any agent starts, the driver checks these. Each failed check prints one l
23
23
 
24
24
  A run that stopped is continued with the same command. `state.json` is the only input of a resume.
25
25
 
26
- One driver runs in a repository at a time. The driver holds a kernel file lock on `implement-spec.lock` in the git common directory (`git rev-parse --git-common-dir`) for the whole run. Each worktree of the repository shares that directory, so a second driver in the same worktree or in another one ends with exit 3 and names the lock file. The file is not in a worktree, so `git status` never shows it. The lock ends with its process, so a driver that was killed leaves no lock behind. `--dry-run` takes no lock. A wrong call in a worktree that has no `.agent/implement/` takes no lock and makes no file; where an earlier run left that folder, the driver takes the lock first, then makes the checks.
26
+ One driver runs on a PR branch at a time. The driver holds a kernel file lock on `implement-spec/<key>.lock` in the git common directory (`git rev-parse --git-common-dir`) for the whole run. `<key>` is the PR branch as one name component: `%` becomes `%25` and `/` becomes `%2F`, so `feat/x` gives `feat%2Fx`. Each worktree of the repository shares that directory, so a second driver on the same PR branch, in the same worktree or in another one, ends with exit 3 and names the lock file. Drivers on different PR branches run at the same time: each ticket branch is `ticket/<key>/<NN>` and the recovery ref is `refs/implement-spec/<key>/recovery`, so two runs share no ref. Git refs are a tree, and one component per PR branch keeps the refs of one run from being a folder of the refs of another. The file is not in a worktree, so `git status` never shows it. The lock ends with its process, so a driver that was killed leaves no lock behind. `--dry-run` takes no lock. A wrong call in a worktree that has no `.agent/implement/` takes no lock and makes no file; where an earlier run left that folder, the driver takes the lock first, then makes the checks.
27
+
28
+ A driver before this change named the ticket branches `ticket/<NN>` and the recovery ref `refs/implement/recovery`. An older driver holds `implement-spec.lock` in the git common directory alone for its whole run. Each newer driver also holds that file, shared, for its whole run: newer drivers do not stop each other, but while an older driver runs in the repository, a newer one ends with exit 3 and names that lock, and while a newer one runs, no older driver starts. A run that the newer driver started has `schema` 2 in `state.json`. When the newer driver resumes a run with no `schema`, it renames each `ticket/<NN>` that holds the start commit of the run and the tip that the run recorded, and moves `refs/implement/recovery` when it holds the recovery point of the run. It writes one ledger sentence for each, and then `schema`. A ref that git cannot rename ends the run with exit 3 and git's error, and nothing is lost.
27
29
 
28
30
  ## Flags
29
31
 
@@ -56,7 +58,7 @@ Default timeouts in minutes: explorer 30, test-writer 30, implementer 90, review
56
58
  | 0 | Done: every ticket is merged and no question is open. Also the end of a valid `--dry-run`. In local mode without `--push`, nothing is pushed. |
57
59
  | 1 | A ticket is BLOCKED, or the end of run is BLOCKED. The other tickets ran as far as they could. |
58
60
  | 2 | Parked: a question of kind `locked-test` or `final-finding` is open. Exit 2 wins over exit 1. |
59
- | 3 | Invalid input or a failed precondition: a usage error, a ticket error, a model error, an uncommitted change, a slot that cannot be made, another driver that runs in the same repository. No agent starts. The one exception: a review page that cannot be written gives exit 3 at the end of a run, after agents ran. |
61
+ | 3 | Invalid input or a failed precondition: a usage error, a ticket error, a model error, an uncommitted change, a slot that cannot be made, another driver that runs on the same PR branch. No agent starts. The one exception: a review page that cannot be written gives exit 3 at the end of a run, after agents ran. |
60
62
  | 4 | Stopped: the CLI of an agent failed two times and wrote no report (for example, the account cannot use the model). The driver starts no other agent and lets the running agents end. No ticket is BLOCKED by it: fix the CLI or the model entry, then run the same command again to resume. |
61
63
  | 130 | The operator interrupted the run. The driver stops each agent. The work stays in the slots for the next run. |
62
64
 
@@ -75,7 +77,7 @@ Everything is under `.agent/`, which is never committed.
75
77
  | Path | Content |
76
78
  |---|---|
77
79
  | `.agent/implement/state.json` | The resume contract: the run record and one record for each ticket (`driver_state.py`). |
78
- | `<git common directory>/implement-spec.lock` | An empty file, not in the worktree. The driver that runs holds a kernel file lock on it (`driver_state.py` `lock`). Do not remove it during a run. |
80
+ | `<git common directory>/implement-spec/<key>.lock` | An empty file, not in the worktree. The driver that runs holds a kernel file lock on it (`driver_state.py` `lock`). Do not remove it during a run. |
79
81
  | `.agent/implement/ledger.md` | One plain sentence for each event. Only the driver writes it, and it reads no fact from it. |
80
82
  | `.agent/implement/summary.md` | The result in sentences, and the measurement table: gates, agents, wall time, fix rounds, human stops, cost. |
81
83
  | `.agent/implement/decisions.json` | The open questions and their `revision`. Written only after a stop. |
@@ -88,7 +90,8 @@ Everything is under `.agent/`, which is never committed.
88
90
  | `.agent/implement-review.html` | The page with the open questions. |
89
91
  | `.agent/implement-answers.json` | The answers: `{"v": 1, "stage": "implement", "submitted": …, "revision": …, "answers": {…}}`. |
90
92
  | `.agent/worktrees/slot-K/` | Slots 2..N. Slot 1 is the PR worktree. |
91
- | `refs/implement/recovery` | A git ref: the PR tip before the tidy. |
93
+ | `refs/implement-spec/<key>/recovery` | A git ref: the PR tip before the tidy. |
94
+ | `ticket/<key>/<NN>` | The branch of one ticket. The driver deletes it when the ticket is merged. |
92
95
 
93
96
  ## Roles of one ticket
94
97
 
@@ -97,10 +100,11 @@ A ticket is ready when each ticket in its **Blocked by** line is merged and no r
97
100
  1. **test-writer** (`prompts/test-writer.md`). It writes one black-box test for each criterion through the Seam and commits them red. Those files are locked. A ticket with Seam `none — refactor` has no test-writer.
98
101
  2. **implementer** (`prompts/implementer.md`). It starts from that commit and makes the locked tests green. It makes the gate (`--gate`) green on its final commit before it reports DONE or DONE_WITH_CONCERNS.
99
102
  3. **Gate.** The driver runs `--gate` on the ticket tip.
100
- 4. **Review.** The driver compares each locked file of the ticket with the test-writer commit. A file that differs is a critical finding `locked file changed: <path>`. Then the **reviewer** (`prompts/reviewer.md`) runs, and for a ticket with a **Risk** line the **correctness** pass (`prompts/correctness.md`) runs at the same time in the same slot.
103
+ 4. **Review.** The driver compares each locked file of the ticket with the test-writer commit. A file that differs is a critical finding `locked file changed: <path>`. A locked file of a merged ticket that the ticket changed since its merge base with the PR branch is a critical `fixture` finding `the ticket changed the locked file of a merged ticket: <path>`, so it goes to a correction (see **Before a correction**). Then the **reviewer** (`prompts/reviewer.md`) runs, and for a ticket with a **Risk** line the **correctness** pass (`prompts/correctness.md`) runs at the same time in the same slot.
101
104
  5. **Merge.** The driver merges the reviewed tip into the PR branch with git plumbing, then deletes the ticket branch. A conflict goes to the **conflict** agent (`prompts/conflict.md`) once. Its resolution may change no locked file of the ticket. A second conflict blocks the ticket.
102
- **The landing check.** Merges are serial. Before the PR branch moves, the driver compares the tree that is about to land (the ticket tip of a fast-forward, or the merge commit) with the PR tip: each locked file of each merged ticket must be the same tree entry in both: the same blob with the same mode, so a file that became a symlink is a difference. A file that only one of them has is a difference. A file that the landing ticket lists in its own `locked` is not checked: the test-writer of a later ticket can add a test to a file that an earlier ticket locked, and must list that file. A test-writer report that changes such a file and does not list it is rejected, with the usual one rerun. A difference refuses the landing: the ticket is BLOCKED with `locked file changed: <path>` in the ledger, the PR branch does not move, and no fix round runs. The PR branch moves only from the tip that the check read; when another hand moved the branch in between, the driver makes the merge, the check and the gate again, one time. Only that case starts the landing again: a refused landing and a red merge gate block the ticket. The second try runs no conflict agent, so a conflict there blocks the ticket, and so does a branch that moved again.
105
+ **The landing check.** Merges are serial. Before the PR branch moves, the driver compares the tree that is about to land (the ticket tip of a fast-forward, or the merge commit) with the PR tip: each locked file of each merged ticket must be the same tree entry in both: the same blob with the same mode, so a file that became a symlink is a difference. A file that only one of them has is a difference. A file that the landing ticket lists in its own `locked` is not checked: the test-writer of a later ticket can add a test to a file that an earlier ticket locked, and must list that file. A test-writer report that changes such a file and does not list it is rejected, with the usual one rerun. The review finds such a change first; the landing check catches a change that no review saw, for example of a ticket that merged after the last review. A difference refuses the landing: the ticket is BLOCKED with `locked file changed: <path>` in the ledger, the PR branch does not move, and no fix round runs. The PR branch moves only from the tip that the check read; when another hand moved the branch in between, the driver makes the merge, the check and the gate again, one time. Only that case starts the landing again: a refused landing and a red merge gate block the ticket. The second try runs no conflict agent, so a conflict there blocks the ticket, and so does a branch that moved again.
103
106
  **Before a correction.** A test-writer that corrects locked tests (after `overrule-test`, or for a `fixture` finding or a `test` finding on a locked file) starts from the locked versions: the driver puts back each locked file of the ticket and each locked file of a merged ticket that the ticket changed, commits them in one commit of its own, and the ledger names each file. A file of a merged ticket goes back to the version of the PR branch. The commit keeps the tree of the next fixer and of the gate clean, whatever files the correction commits. The correction's prompt names the restored files: when a test in one of them fails only because the ticket changed an interface that the test uses, the correction updates the file to the new interface, keeps what each test asserts, and lists the file in `locked`.
107
+ **The gate of the restore.** In the fix loop, the tip before the restore has a green gate, so the driver gates the restore commit and does not leave the question to the correction. A red gate means that a restored file fails with the ticket's code: the correction gets the gate record (`prompts/restored-red.md`), and a DONE correction must list a restored file in `locked`; else it reports BLOCKED with `locked test contradicts spec:`. A green gate with only driver findings to correct closes them with the restore: no test-writer runs. After `overrule-test` the driver does not gate the restore, because the tip before it has no green gate; a change that the next implementer makes to a merged ticket's file is a finding of the next review.
104
108
 
105
109
  Rules for each agent:
106
110
 
@@ -130,7 +134,7 @@ A red gate on the commit of the implementer or of a fixer is a critical finding
130
134
 
131
135
  The end of run starts only when every ticket is merged and every slot is idle. Its agents work in slot 1.
132
136
 
133
- 1. **Tidy.** The driver saves the PR tip in `refs/implement/recovery`. It then makes one commit for each ticket, in merge order, named `feat(NN): <title>`. Each commit holds the tests, the implementation and the fixes of its ticket. The tree of the new tip must equal the tree of the old tip, or the branch stays as it was.
137
+ 1. **Tidy.** The driver saves the PR tip in `refs/implement-spec/<key>/recovery`. It then makes one commit for each ticket, in merge order, named `feat(NN): <title>`. Each commit holds the tests, the implementation and the fixes of its ticket. The tree of the new tip must equal the tree of the old tip, or the branch stays as it was.
134
138
  2. **Rebase** onto the fetched tip of the default branch of `origin`. A conflict goes to the conflict agent (`prompts/rebase-conflict.md`) once, and the rebased tip is gated.
135
139
  3. **Gate**, then the **final-reviewer** (`prompts/final-reviewer.md`) on the whole range, with the method of the `review-diff` skill. It gets the minor findings of the tickets and the rulings of the operator.
136
140
  4. **Fix batch.** Critical and important findings go to one fixer (`prompts/final-fixer.md`). The driver names each of its commits `fix(NN): …` from the ticket in the report. Then the gate, then one scoped review of those commits.
@@ -172,7 +176,7 @@ Every report has `status` (`DONE`, `DONE_WITH_CONCERNS`, `NEEDS_CONTEXT`, `BLOCK
172
176
 
173
177
  | Role | Template | More fields | Checks after the schema |
174
178
  |---|---|---|---|
175
- | test-writer | `prompts/test-writer.md` | `locked` (test file paths), `red` (`test`, `line`) | The driver reads a test node id `<file>::<test>` in `locked` as its file. With DONE: one commit, one locked file and one red line or more. Each locked path is in a commit that the role made. Each `line` is in the log. A locked file of a merged ticket that the role changed is in `locked`. A correction (after `overrule-test`, or for a review finding) changes only the files that it lists in `locked` and the files that the driver restored before it. |
179
+ | test-writer | `prompts/test-writer.md` | `locked` (test file paths), `red` (`test`, `line`) | The driver reads a test node id `<file>::<test>` in `locked` as its file. With DONE: one commit, one locked file and one red line or more. Each locked path is in a commit that the role made. Each `line` is in the log. A locked file of a merged ticket that the role changed is in `locked`. A correction (after `overrule-test`, or for a review finding) changes only the files that it lists in `locked` and the files that the driver restored before it. When the gate is red on the restore commit, a DONE correction lists a restored file in `locked`. |
176
180
  | implementer | `prompts/implementer.md` | `green` | With DONE: one commit or more. |
177
181
  | reviewer | `prompts/reviewer.md` | `findings` (`severity`, `file`, `line`, `title`, `evidence`, `kind`), `mutations` (`test`, `file`, `line`, `red`); `commits` is empty | In the first review of a tip: `mutations` is required, with a proof for each locked test file (`test` is `<locked file>::<test name>`). Each `red` is in the log. |
178
182
  | correctness | `prompts/correctness.md` | the same as the reviewer; `commits` is empty | `mutations` is optional. |
@@ -202,7 +206,7 @@ Answer with `--answer <question id>=<option id>` on the next run, or through the
202
206
 
203
207
  | Check | Where it lives |
204
208
  |---|---|
205
- | Locked tests with a demonstrated red | `driver_report.py` `_proof_errors`: each red line of the test-writer report must be in its log. `driver_review.py` `locked_findings`: a locked file of the ticket that differs from the test-writer commit is a critical finding, before each review. `driver_merge.py` `refused`, called from `driver.py` `_checked_landing` under the merge lock: a locked file of a merged ticket that differs between the PR tip and the tree that is about to land refuses the landing, so a later ticket cannot change the locked test of an earlier one. `driver_report.py` `_merged_errors` rejects a test-writer report that changes such a file and does not list it. |
209
+ | Locked tests with a demonstrated red | `driver_report.py` `_proof_errors`: each red line of the test-writer report must be in its log. `driver_review.py` `locked_findings`: a locked file of the ticket that differs from the test-writer commit is a critical finding, before each review. `driver_review.py` `merged_changes`: a locked file of a merged ticket that the ticket changed is a critical `fixture` finding, before each review. `driver_merge.py` `refused`, called from `driver.py` `_checked_landing` under the merge lock: a locked file of a merged ticket that differs between the PR tip and the tree that is about to land refuses the landing, so a later ticket cannot change the locked test of an earlier one. `driver_report.py` `_merged_errors` rejects a test-writer report that changes such a file and does not list it. |
206
210
  | One mutation proof for each locked test | `prompts/reviewer.md` asks for one proof for each locked test. `driver_report.py` `_locked_errors` and `_proof_errors` reject a first review with no proof for a locked test file, or with a red line that is not in the log. `driver_review.py` `follow` finds the proofs that a fix makes due again. |
207
211
  | One independent correctness pass for each risky ticket | `driver_fixloop.py` `_review` adds the pass for a ticket with a **Risk** line. `driver_agents.py` `correctness_entry` picks the entry of another provider, and `load_models` stops the run when there is none or its CLI is not on PATH. |
208
212
  | One full gate for each candidate | `driver_gate.py` `run`, called from `driver_run.py` `Run.gate`: one record for each commit the driver is about to trust. Green is exit 0 on a tree that is clean before and after, with HEAD not moved. |
@@ -14,8 +14,10 @@ from __future__ import annotations
14
14
  import argparse
15
15
  import json
16
16
  import os
17
+ import re
17
18
  import sys
18
19
  from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
20
+ from contextlib import ExitStack
19
21
  from dataclasses import dataclass
20
22
  from pathlib import Path
21
23
 
@@ -25,6 +27,7 @@ import driver_decisions
25
27
  import driver_final
26
28
  import driver_fixloop
27
29
  import driver_graph
30
+ import driver_legacy
28
31
  import driver_merge
29
32
  import driver_review
30
33
  import driver_review_page
@@ -32,6 +35,7 @@ import driver_slots
32
35
  import driver_state
33
36
  import driver_summary
34
37
  import driver_tickets
38
+ from driver_names import Names
35
39
  from driver_run import FINAL, OUT, Blocked, Halted, Run, during, ledger
36
40
  from driver_slots import git
37
41
  from driver_tickets import Ticket
@@ -59,7 +63,7 @@ def _roles_left(ticket: Ticket, phase: str, only: str | None) -> list[str]:
59
63
 
60
64
  def _check_out_ticket(ticket: Ticket, slot_number: int, entry: dict, run: Run) -> str:
61
65
  """Put the ticket branch in the slot and return the commit its next role starts from."""
62
- slot, branch = driver_slots.path(slot_number), f"ticket/{ticket.number}"
66
+ slot, branch = driver_slots.path(slot_number), run.names.ticket(ticket.number)
63
67
  if run.kept.get(ticket.number) == slot_number:
64
68
  # The slot holds the uncommitted work of an interrupted role: the branch and the tree stay as they are.
65
69
  return entry["tip"] or git(slot, "merge-base", "HEAD", run.tip()).stdout.strip()
@@ -80,7 +84,8 @@ def _prompt_values(ticket: Ticket, slot: str, run: Run) -> dict[str, str]:
80
84
  """The pointers every prompt of the ticket gets. An agent works in its slot, so paths are relative to it."""
81
85
  folder = OUT / ticket.number
82
86
  return {"spec": os.path.relpath(run.args.spec, slot), "seam": ticket.seam, "pr_branch": run.pr,
83
- "branch": f"ticket/{ticket.number}", "ticket": os.path.relpath(Path(run.args.tickets) / ticket.file, slot),
87
+ "branch": run.names.ticket(ticket.number),
88
+ "ticket": os.path.relpath(Path(run.args.tickets) / ticket.file, slot),
84
89
  "correctness": str(driver_review.CORRECTNESS), "gate": run.args.gate,
85
90
  "blocked_report": os.path.relpath(_blocked_report(ticket.number, run), slot),
86
91
  "writer_report": os.path.relpath(folder / "test-writer-0.json", slot)}
@@ -152,7 +157,7 @@ def _resolve_conflict(ticket: Ticket, slot: str, run: Run, tip: str, files: list
152
157
 
153
158
  def _merge_commit(ticket: Ticket, slot: str, run: Run, tip: str, may_resolve: bool) -> _Landing:
154
159
  """Merge the ticket tip with the PR tip into a commit that no ref holds yet."""
155
- message = f"Merge ticket/{ticket.number} into {run.pr}"
160
+ message = f"Merge {run.names.ticket(ticket.number)} into {run.pr}"
156
161
  pr_tip = run.tip()
157
162
  commit, error = driver_merge.commit(pr_tip, tip, message)
158
163
  if not error:
@@ -171,7 +176,8 @@ def _merge_commit(ticket: Ticket, slot: str, run: Run, tip: str, may_resolve: bo
171
176
  def _checked_landing(ticket: Ticket, slot: str, run: Run, may_resolve: bool) -> _Landing:
172
177
  """The commit to land, with the locked-file check passed and the gate green. Call it under the merge lock."""
173
178
  number = ticket.number
174
- git(slot, "checkout", "-q", f"ticket/{number}") # an earlier try left the slot on its merge commit
179
+ # An earlier try left the slot on its merge commit.
180
+ git(slot, "checkout", "-q", run.names.ticket(number))
175
181
  tip = run.state.ticket(number, phase="merge")["tip"]
176
182
  landing = _merge_commit(ticket, slot, run, tip, may_resolve)
177
183
  refusal = driver_merge.refused(run.state.read()["tickets"], number, landing.pr_tip, landing.commit)
@@ -208,7 +214,7 @@ def _record_landing(state: dict, number: str, pr_tip: str) -> None:
208
214
 
209
215
  def _merge(ticket: Ticket, slot: str, run: Run) -> None:
210
216
  """Land the reviewed ticket tip on the PR branch, then delete the ticket branch."""
211
- number, branch = ticket.number, f"ticket/{ticket.number}"
217
+ number, branch = ticket.number, run.names.ticket(ticket.number)
212
218
  # The merge gate and the conflict agent run under the lock: the commit the gate saw is the one that lands.
213
219
  with run.merging, during("the merge"):
214
220
  landing = _land(ticket, slot, run)
@@ -348,27 +354,16 @@ def _work(tickets: list[Ticket], slots: int, run: Run) -> tuple[bool, list[str]]
348
354
 
349
355
  # ---------------------------------------------------------------- before the first agent: resume and answers
350
356
 
351
- def _pr_branch(state: driver_state.State) -> tuple[str, str]:
352
- """(the PR branch, "") or ("", why the run cannot start)."""
353
- here = git(".", "symbolic-ref", "-q", "--short", "HEAD").stdout.strip()
354
- # An interrupted run leaves slot 1 on a ticket branch or detached, so the record names the PR branch.
355
- pr = state.read()["run"]["pr_branch"] if state.path.exists() else here
356
- if not pr:
357
- return "", "the PR worktree is on a detached HEAD: check out the PR branch"
358
- if here not in ("", pr) and not here.startswith("ticket/"):
359
- return "", f"{state.path} records a run on {pr}, not on {here}: check out {pr}, or remove {OUT}"
360
- return pr, ""
361
-
362
-
363
357
  def _resume_error(number: str, state: driver_state.State) -> str:
364
358
  """Check the record of one ticket against git. A ticket that git contradicts goes back to its last verified
365
359
  phase; "" or the error that stops the run."""
366
- entry, branch = state.read()["tickets"][number], f"ticket/{number}"
360
+ run, entry = state.read()["run"], state.read()["tickets"][number]
361
+ branch = Names(run["pr_branch"]).ticket(number)
367
362
  if entry["phase"] not in driver_state.PHASES:
368
363
  return f"{state.path}: ticket {number} has the unknown phase {entry['phase']!r}"
369
- if entry["phase"] == "pending" and git(".", "rev-parse", "-q", "--verify", f"refs/heads/{branch}").returncode == 0:
364
+ if entry["phase"] == "pending" and driver_slots.branch_exists(branch):
370
365
  return f"branch {branch} exists and {state.path} does not record it: delete or rename the branch"
371
- fields, why = driver_state.verified(entry, branch, state.read()["run"]["pr_branch"])
366
+ fields, why = driver_state.verified(entry, branch, run["pr_branch"])
372
367
  if fields:
373
368
  state.ticket(number, **fields)
374
369
  start = fields["tip"][:7] if fields["tip"] else "the PR tip"
@@ -376,15 +371,12 @@ def _resume_error(number: str, state: driver_state.State) -> str:
376
371
  return ""
377
372
 
378
373
 
379
- def _resume(tickets: list[Ticket], state: driver_state.State, slots: int, args, kept: dict) -> tuple[str, list[str]]:
380
- """Start or resume state.json and make the slots: (the PR branch, the errors that stop the run)."""
381
- pr, error = _pr_branch(state)
382
- if error:
383
- return "", [error]
374
+ def _resume(tickets: list[Ticket], state: driver_state.State, pr: str, slots: int, args, kept: dict) -> list[str]:
375
+ """Start or resume state.json on the PR branch `pr` and make the slots: the errors that stop the run."""
384
376
  state.start(pr, [ticket.number for ticket in tickets])
385
377
  driver_final.finish_interrupted_move(state, pr)
386
378
  errors = [error for error in (_resume_error(ticket.number, state) for ticket in tickets) if error]
387
- return pr, errors or driver_slots.setup(slots, args.prepare, set(kept.values()))
379
+ return errors or driver_slots.setup(slots, args.prepare, set(kept.values()))
388
380
 
389
381
 
390
382
  def _publish_questions(run: dict) -> str:
@@ -474,10 +466,10 @@ def _exit_code(final: dict, args, end_blocked: bool) -> int:
474
466
  return code
475
467
 
476
468
 
477
- def _run(tickets: list[Ticket], models: dict[str, list[str]], args, kept: dict[str, int]) -> int:
469
+ def _run(tickets: list[Ticket], models: dict[str, list[str]], args, kept: dict[str, int], pr: str) -> int:
478
470
  state = driver_state.State(STATE)
479
471
  slots = 1 if args.only else min(args.parallel, len(tickets))
480
- pr, errors = _resume(tickets, state, slots, args, kept)
472
+ errors = _resume(tickets, state, pr, slots, args, kept)
481
473
  errors = errors or _apply_answers(state, args)
482
474
  for error in errors:
483
475
  print(error)
@@ -564,7 +556,7 @@ def _validated(args: argparse.Namespace) -> tuple[list[Ticket], dict[str, list[s
564
556
  errors.append("--spec is required: the spec file the prompts point at")
565
557
  if args.parallel < 1:
566
558
  errors.append("--parallel must be 1 or more")
567
- kept = driver_state.kept(json.loads(STATE.read_text())) if STATE.exists() else {}
559
+ kept = driver_state.kept(driver_state.State(STATE).read()) if STATE.exists() else {}
568
560
  errors += _worktree_errors(1 in kept.values())
569
561
  models, model_errors = driver_agents.load_models(args.models)
570
562
  tickets, ticket_errors = driver_tickets.parse(args.tickets)
@@ -579,16 +571,61 @@ def _print_width(tickets: list[Ticket]) -> None:
579
571
  print(f"max frontier width: {driver_tickets.max_frontier_width(tickets)}")
580
572
 
581
573
 
582
- def _lock_file(args: argparse.Namespace) -> Path | None:
583
- """The file one driver per repository locks: in the git common directory, which every worktree shares and
584
- `git status` never reports. None, with each error printed, when the call is not valid."""
574
+ def _run_branch() -> str:
575
+ """The PR branch the run in this worktree is on: the one state.json records, else the one checked out."""
576
+ # HEAD first: a driver writes state.json before it moves HEAD, so a HEAD that a run moved has a state.json.
577
+ here = driver_slots.on_branch(".")
578
+ return driver_state.State(STATE).read()["run"]["pr_branch"] if STATE.exists() else here
579
+
580
+
581
+ def _start_errors(pr: str) -> list[str]:
582
+ """Under the lock of `pr`, before any check reads a ref: the errors that stop the run, or [].
583
+
584
+ The run of this worktree must still be on `pr`, a run that an older driver started gets its new names first,
585
+ and slot 1 must be on `pr` or where an interrupted run left it: on one of its ticket branches, or detached."""
586
+ if _run_branch() != pr:
587
+ return [f"the run in this worktree left {pr} while the driver took its lock: run the driver again"]
588
+ if errors := driver_legacy.resume(driver_state.State(STATE)):
589
+ return errors
590
+ here = driver_slots.on_branch(".")
591
+ # An older driver named its ticket branches `ticket/NN`, and a branch that the resume does not rename stays so.
592
+ if here in ("", pr) or here.startswith(Names(pr).ticket("")) or re.fullmatch(r"ticket/\d+", here):
593
+ return []
594
+ return [f"{STATE} records a run on {pr}, not on {here}: check out {pr}, or remove {OUT}"]
595
+
596
+
597
+ def _common_dir_and_branch(args: argparse.Namespace) -> tuple[Path, str] | None:
598
+ """(the git common directory, which every worktree shares and `git status` never reports, and the PR branch of
599
+ the run): what names the lock. None, with each error printed, when the call is not valid."""
585
600
  common = git(".", "rev-parse", "--git-common-dir")
586
601
  # A first call is validated before the lock, so a wrong call makes no file. After an earlier run the lock comes
587
602
  # first: the driver that holds it changes what the validation reads.
588
603
  first_call = common.returncode or not OUT.exists()
589
604
  if first_call and not _validated(args):
590
605
  return None
591
- return Path(common.stdout.strip()).resolve() / "implement-spec.lock"
606
+ if not (pr := _run_branch()):
607
+ print("the PR worktree is on a detached HEAD: check out the PR branch")
608
+ return None
609
+ if pr.startswith("ticket/"): # a slot of another run: its driver owns the worktree and the branch
610
+ print(f"the PR worktree is on the ticket branch {pr} of a run: check out the PR branch")
611
+ return None
612
+ return Path(common.stdout.strip()).resolve(), pr
613
+
614
+
615
+ def _hold(common: Path, pr: str) -> tuple[ExitStack | None, str]:
616
+ """Take the locks of a run on `pr`: (the stack that holds them until it closes, "") or (None, why another
617
+ driver stops this one)."""
618
+ with ExitStack() as stack:
619
+ lock_file = Names(pr).lock_file(common)
620
+ if not (held := driver_state.lock(lock_file)):
621
+ return None, (f"another driver runs on {pr}: it holds the lock {lock_file}. Wait until it ends, then "
622
+ "run the driver again.")
623
+ stack.enter_context(held)
624
+ older, error = driver_legacy.lock(common)
625
+ if not older:
626
+ return None, error
627
+ stack.enter_context(older)
628
+ return stack.pop_all(), ""
592
629
 
593
630
 
594
631
  def main(argv: list[str] | None = None) -> int:
@@ -597,15 +634,19 @@ def main(argv: list[str] | None = None) -> int:
597
634
  if valid := _validated(args):
598
635
  _print_width(valid[0])
599
636
  return 0 if valid else 3
600
- if not (lock_file := _lock_file(args)):
637
+ if not (locking := _common_dir_and_branch(args)):
601
638
  return 3
602
- held = driver_state.lock(lock_file)
639
+ common, pr = locking
640
+ held, error = _hold(common, pr)
603
641
  if not held:
604
- print(f"implement-spec: another driver runs in this repository: it holds the lock {lock_file}. Wait until it "
605
- "ends, then run the driver again.")
642
+ print(f"implement-spec: {error}")
606
643
  return 3
607
644
  with held:
608
- # Only what is read under the lock is trusted: no other driver changes state.json, the slots or the refs.
645
+ # Only what is read under the lock is trusted: no other driver changes state.json, the slots, the PR branch
646
+ # or its ticket branches. Drivers on other PR branches touch none of them.
647
+ if errors := _start_errors(pr):
648
+ print("".join(f"implement-spec: {error}\n" for error in errors), end="")
649
+ return 3
609
650
  if not (valid := _validated(args)):
610
651
  return 3
611
652
  tickets, models, kept = valid
@@ -614,7 +655,7 @@ def main(argv: list[str] | None = None) -> int:
614
655
  print("".join(error + "\n" for error in errors), end="")
615
656
  return 3
616
657
  _print_width(tickets)
617
- return _run(tickets, models, args, kept)
658
+ return _run(tickets, models, args, kept, pr)
618
659
 
619
660
 
620
661
  if __name__ == "__main__":
@@ -4,7 +4,7 @@ commits with a scoped review, then stop before the push, or push and open the PR
4
4
 
5
5
  `run.final` of state.json is the resume contract (`driver_state._final` makes it with every key):
6
6
  skipped why the end-of-run work did not start
7
- recovery the PR tip before the rewrite; the ref RECOVERY holds it. None when origin had the history already
7
+ recovery the PR tip before the rewrite; Run.names names its ref. None when origin had the history already
8
8
  tidied the tip the rewrite made, or the tip as the tickets left it when the history is kept
9
9
  base, onto the default branch of origin and its tip under the history; with no origin: "" and the start of the run
10
10
  commits {ticket: its commit}
@@ -34,7 +34,6 @@ from driver_slots import git, head
34
34
  from driver_state import State
35
35
  from driver_tickets import Ticket
36
36
 
37
- RECOVERY = "refs/implement/recovery"
38
37
  KIND = "final-finding"
39
38
  ROUNDS = 2 # the full review and the scoped one after the fix batch
40
39
  # The final review runs through the review-diff skill, which is installed next to this one.
@@ -170,7 +169,8 @@ def _resolve_rebase_conflict(run: Run, env: dict, base: str, onto: str) -> None:
170
169
  raise Blocked(f"the conflict agent: {error}" if error else
171
170
  f"the rebase onto {base} conflicts a second time, in {', '.join(again)}")
172
171
  # The resolution may not change a locked test: each one is as the recovery point has it.
173
- changed = driver_review.locked_changes(".", git(".", "rev-parse", RECOVERY).stdout.strip(), _locked_files(run))
172
+ before = git(".", "rev-parse", run.names.recovery).stdout.strip()
173
+ changed = driver_review.locked_changes(".", before, _locked_files(run))
174
174
  if error := error or "; ".join(finding["title"] for finding in changed):
175
175
  raise Blocked(f"the conflict agent: {error}")
176
176
  ledger(f"Rebase: the conflict agent resolved the conflict with {base} in {', '.join(files)} at {head('.')[:7]}.")
@@ -249,7 +249,7 @@ def _rewrite_once(run: Run, tickets: list[Ticket]) -> None:
249
249
  final = run.state.read()["run"]["final"]
250
250
  if final["tidied"]:
251
251
  if not driver_merge.ancestor(final["tidied"], run.tip()):
252
- raise Blocked(f"{run.pr} does not hold the tidied tip {final['tidied'][:7]}; {RECOVERY} holds the "
252
+ raise Blocked(f"{run.pr} does not hold the tidied tip {final['tidied'][:7]}; {run.names.recovery} holds the "
253
253
  "history before the tidy")
254
254
  return
255
255
  base, onto = _base(run)
@@ -258,9 +258,9 @@ def _rewrite_once(run: Run, tickets: list[Ticket]) -> None:
258
258
  _keep_published_history(run, old, base, onto)
259
259
  return
260
260
  tree = git(".", "rev-parse", f"{old}^{{tree}}").stdout.strip()
261
- git(".", "update-ref", "-m", "implement-spec: recovery point", RECOVERY, old)
261
+ git(".", "update-ref", "-m", "implement-spec: recovery point", run.names.recovery, old)
262
262
  ledger(f"Recovery point: {run.pr} was at {old} with the tree {tree} before the rebase and the tidy; "
263
- f"{RECOVERY} holds that commit.")
263
+ f"{run.names.recovery} holds that commit.")
264
264
  git(".", "checkout", "-q", "--detach") # slot 1 is off the branch while the history is made
265
265
  try:
266
266
  commits = _new_history(run, tickets, old, base, onto)
@@ -11,6 +11,7 @@ from concurrent.futures import ThreadPoolExecutor
11
11
  from dataclasses import dataclass, field
12
12
 
13
13
  import driver_agents
14
+ import driver_merge
14
15
  import driver_review
15
16
  import driver_slots
16
17
  from driver_run import OUT, Blocked, Run, during, ledger
@@ -86,7 +87,8 @@ def _review(job: Job) -> list[dict]:
86
87
  """Review `job.tip`. Returns the findings of the driver and of each reviewer."""
87
88
  run, number = job.run, job.ticket.number
88
89
  state = run.state.ticket(number, phase="review")
89
- found = _log_findings(job, driver_review.locked_findings(job.slot, state))
90
+ merged = driver_merge.changed(job.slot, run.tip(), driver_merge.protected(run.state.read()["tickets"], number))
91
+ found = _log_findings(job, driver_review.locked_findings(job.slot, state) + driver_review.merged_changes(merged))
90
92
  cast = [("reviewer", "")]
91
93
  if job.ticket.risk:
92
94
  cast.append(("correctness", driver_agents.correctness_entry(run.models)))
@@ -107,11 +109,21 @@ def _correct(job: Job, findings: list[dict]) -> None:
107
109
  raise Blocked(f"the review: a locked-test finding is open after the test-writer correction: {titles}")
108
110
  restored = run.restore_locked_files(number, job.slot)
109
111
  job.tip = driver_slots.head(job.slot) # the restore is a commit of the driver
110
- values = {**job.pointers(job.round), "restored": " ".join(restored)}
111
- notes = ["correction"] + ["restored"] * bool(restored)
112
+ # The gate was green on the tip before the restore, so a red gate here is a restored file that fails with the
113
+ # ticket's code. The driver decides that, not the correction (issue #90).
114
+ with during("the gate of the restore"):
115
+ record = run.red_gate(number, job.slot, job.tip) if restored else ""
116
+ if not record and all(finding["source"] == "driver" for finding in findings):
117
+ job.previous = "" # no review has seen the restore commit
118
+ ledger(f"Ticket {number}: the code does not need the change to {', '.join(restored)}: the restore closes it, "
119
+ "with no correction.")
120
+ return
121
+ values = {**job.pointers(job.round), "restored": " ".join(restored),
122
+ "record": record and os.path.relpath(record, job.slot)}
123
+ notes = ["correction"] + ["restored"] * bool(restored) + ["restored-red"] * bool(record)
112
124
  with during("the test-writer correction"):
113
125
  report = run.agent("test-writer", number, job.slot, values, notes=notes, fix_round=job.round + 1,
114
- check={"correction": True, "restored": restored})
126
+ check={"correction": True, "restored": restored, "restore_red": bool(record)})
115
127
  job.tip, job.previous, job.corrected = driver_slots.head(job.slot), "", True
116
128
  entry = run.state.read()["tickets"][number]
117
129
  run.state.ticket(number, tip=job.tip, **driver_review.written(job.slot, entry, report["locked"], job.tip))
@@ -153,7 +165,7 @@ def _act_on(job: Job, found: list[dict]) -> bool:
153
165
  # No correction after round 4: no fix round could make its new locked tests green, so `accept` could not merge.
154
166
  to_correct, to_fix = [], [finding for finding in found if finding in to_correct + to_fix]
155
167
  # A changed locked file is fixed first: a correction on top of it would make the changed file the reference.
156
- locked_file_changed = any(finding["source"] == "driver" for finding in found)
168
+ locked_file_changed = any(finding["source"] == "driver" for finding in to_fix)
157
169
  if to_correct and not locked_file_changed:
158
170
  _correct(job, to_correct)
159
171
  if not to_fix:
@@ -0,0 +1,92 @@
1
+ """A run that an older driver started: one lock per repository, ticket branches `ticket/NN` and the recovery ref
2
+ `refs/implement/recovery`.
3
+
4
+ Every run of a newer driver holds `lock` (driver._hold), so a newer and an older driver never run in one repository
5
+ at the same time. The driver calls `resume` once, after the locks and before any check reads a ref: a run with no
6
+ `schema` in state.json gets the new names of its refs.
7
+ ponytail: delete this module, tests/test_legacy.py and its one line in driver._hold and in driver._start_errors
8
+ once no older driver is left.
9
+ """
10
+ from __future__ import annotations
11
+
12
+ from pathlib import Path
13
+ from typing import IO
14
+
15
+ import driver_slots
16
+ import driver_state
17
+ from driver_names import Names
18
+ from driver_run import ledger
19
+ from driver_slots import git
20
+
21
+ LOCK = "implement-spec.lock" # in the git common directory: an older driver holds it for its whole run
22
+ RECOVERY = "refs/implement/recovery"
23
+
24
+
25
+ def _checked_out_elsewhere(branch: str) -> bool:
26
+ """True when a worktree that is not this one nor one of its slots has `branch` checked out."""
27
+ here = Path.cwd().resolve()
28
+ for block in git(".", "worktree", "list", "--porcelain").stdout.split("\n\n"):
29
+ lines = block.splitlines()
30
+ if lines and f"branch refs/heads/{branch}" in lines:
31
+ where = Path(lines[0].removeprefix("worktree ")).resolve()
32
+ return where != here and here / ".agent" / "worktrees" not in where.parents
33
+ return False
34
+
35
+
36
+ def _this_runs(entry: dict, legacy: str, start: str) -> bool:
37
+ """A `ticket/NN` is this run's when it holds the commit the run started from and the tip the run recorded, and
38
+ no worktree outside this run has it checked out."""
39
+ if not driver_slots.ancestor(start, legacy) or _checked_out_elsewhere(legacy):
40
+ return False
41
+ return not entry["tip"] or driver_slots.ancestor(entry["tip"], legacy)
42
+
43
+
44
+ def _branch(entry: dict, number: str, names: Names, start: str) -> str:
45
+ """Rename one `ticket/NN` of this run: "" or the error."""
46
+ legacy, new = f"ticket/{number}", names.ticket(number)
47
+ if entry["phase"] in ("pending", "merged") or not driver_slots.branch_exists(legacy):
48
+ return ""
49
+ if driver_slots.branch_exists(new) or not _this_runs(entry, legacy, start):
50
+ return ""
51
+ moved = git(".", "branch", "-m", legacy, new) # git also moves the HEAD of a slot that has the branch
52
+ if moved.returncode:
53
+ return f"cannot rename the branch {legacy} of an older driver to {new}: {moved.stderr.strip()}"
54
+ ledger(f"Ticket {number}: renamed the branch {legacy} of an older driver to {new}.")
55
+ return ""
56
+
57
+
58
+ def _recovery(run: dict, names: Names) -> str:
59
+ """Move `refs/implement/recovery` to the name of this run when it holds the recovery point that this run
60
+ recorded: "" or the error."""
61
+ recorded = run.get("final", {}).get("recovery")
62
+ if not recorded or git(".", "rev-parse", "-q", "--verify", RECOVERY).stdout.strip() != recorded:
63
+ return ""
64
+ # The old ref goes only when the new one holds the commit: a move that stopped half way is finished next time.
65
+ made = git(".", "update-ref", "-m", "implement-spec: rename", names.recovery, recorded)
66
+ if made.returncode:
67
+ return f"cannot move {RECOVERY} of an older driver to {names.recovery}: {made.stderr.strip()}"
68
+ git(".", "update-ref", "-d", RECOVERY, recorded)
69
+ ledger(f"Recovery point: moved {RECOVERY} of an older driver to {names.recovery}.")
70
+ return ""
71
+
72
+
73
+ def lock(common_dir: Path) -> tuple[IO | None, str]:
74
+ """The lock of an older driver, shared: (the held file, "") or (None, the error) while an older driver runs in
75
+ the repository. Newer drivers hold it together, so they do not stop each other; an older driver holds it alone,
76
+ so it never runs beside one."""
77
+ if held := driver_state.lock(common_dir / LOCK, shared=True):
78
+ return held, ""
79
+ return None, (f"a driver of an older version runs in this repository: it holds the lock {common_dir / LOCK}. "
80
+ "Wait until it ends, then run the driver again.")
81
+
82
+
83
+ def resume(state: driver_state.State) -> list[str]:
84
+ """Give each ref of a run that an older driver started its new name, then mark the run: the errors, or []."""
85
+ if not state.path.exists() or state.read()["run"].get("schema", 1) >= driver_state.SCHEMA:
86
+ return []
87
+ run, names = state.read()["run"], Names(state.read()["run"]["pr_branch"])
88
+ moves = [_branch(entry, number, names, run["start"]) for number, entry in state.read()["tickets"].items()]
89
+ if errors := [error for error in [*moves, _recovery(run, names)] if error]:
90
+ return errors
91
+ state.update(lambda changed: changed["run"].update(schema=driver_state.SCHEMA))
92
+ return []