@clemrig/factory 0.9.0 → 0.10.0

package/README.md

@@ -73,14 +73,14 @@ An issue is a folder `.factory/issues/<slug>/` holding at least one of the files
 | `implement`, `implemented` | The `implement` agent receives the spec, the plan and the run's browser, in one conversation for the run, and answers with evidence for each criterion and its deviations from the plan; the run stops if it committed, switched branches or changed `.factory/`, whatever it answered |
 | `verify` | The repository's checks, in order, a check with `when` only if a file changed since the reference commit matches it; a failure, a timeout or a command that cannot start goes back to `implement` with the command, the cause and the end of its output (3 times at most). What each check wrote is kept in the run's folder, for the review, and a failed check's whole output in `checks/failed-<n>-<name>.log`, the run's `n`th failure, resumptions included, whose path the run journals (over 2 MiB per stream, its end, marked as truncated) |
 | `diff` | Every change since the reference commit, new files included, with each file's added and removed lines; the run stops if it is over 2 MiB. Over 1,500 lines of diff, the change is large: its diff is kept in the run's folder instead of being sent in the review's request. |
-| `review`, `reviewed` | The `review` agent, in a new conversation for each review, receives the spec, the changes (the diff, or for a large change where to read it), the checks and where their output is, and the run's browser, never the implementing agent's answer. It judges each criterion `met` or `not_met`, with its decisive evidence, and gives each finding a nature. From the second round on, once it has answered on its own, it receives in the same conversation the previous review: it checks each earlier criterion and finding against the code, lists those now fixed with their proof, keeps those that still hold, with the same nature, judges how they were fixed, and answers its whole review again. Every review judged is kept in the run. An unmet criterion or a blocking finding goes back to `implement` (2 times at most) with the other findings, to fix as well when that is safe, and the implementing agent fixes the mechanism that let a defect happen, wherever the change has it, rather than adding a guard per case |
+| `review`, `reviewed` | The `review` agent, in a new conversation for each review, receives the spec, the changes (the diff, or for a large change where to read it), the checks and where their output is, and the run's browser, never the implementing agent's answer. It judges each criterion `met` or `not_met`, with its decisive evidence, and gives each finding a nature, only on what it established: a test it ran, the application it tried or a code path it followed to its end; a risk it could not establish is at most a `weak_test`. From the second round on, once it has answered on its own, it receives in the same conversation the previous review: it checks each earlier criterion and finding against the code, lists those now fixed with their proof, keeps those that still hold, with the same nature, judges how they were fixed, tries each fix in the situations around it (a fix that breaks what worked is a `regression`), and answers its whole review again. Every review judged is kept in the run. An unmet criterion or a blocking finding goes back to `implement` (4 times at most) with the other findings, to fix as well when that is safe, and the implementing agent fixes the mechanism that let a defect happen, wherever the change has it, rather than adding a guard per case |
 | `describe`, `described` | Once the change is approved, the `describe` agent explains it for the pull request (see below) and takes its screenshots. It receives the spec, the plan, the changes, the checks, the approved review, the deviations from the plan, whether the repository is public, the screenshots folder and the run's browser, never the implementing agent's summary. Its guide must name changed files only, each once, its screenshots must be distinct images of the folder, 10 MB at most, and the pull request's body must stay under 60,000 characters; otherwise its answer goes back to it (2 times at most) |
 | `deliver` | Commits the changes with the plan's conventional commit subject, force-pushes `<slug>` to `origin`, updates the branch's open pull request or opens one with `gh`, then returns to the main branch |
 | `attach` | When there are screenshots: replaces the pull request's body with one that shows them and uploads them (`gh pr edit --attach`) |
 
 A finding's nature alone decides whether it blocks the change: `incorrect_behavior`, `data_loss`, `outage`, `security`, `regression` and `spec_violation` block it; `weak_test`, `maintainability` and `documentation` do not.
 
-If its profile allows it, the reviewer runs commands (tests, reproductions, the application in a browser) and, on a large change or one that spans several applications or layers, splits the review between sub-agents (the `Agent` tool), each taking one part, and one more the paths across them: failures, interruptions, concurrency, recovery. It reports a defect once with every place it occurs, so that it is fixed everywhere at once. Neither the reviewer nor the describing agent changes the repository:
+If its profile allows it, the reviewer runs commands (tests, reproductions, the application in a browser) and, on a large change or one that spans several applications or layers, splits the whole review between sub-agents (the `Agent` tool), keeping no part for itself: each takes one part, its browser checks included, and one more, always, the paths across them: failures, interruptions, concurrency, recovery. It reports a defect once with every place it occurs, so that it is fixed everywhere at once. Neither the reviewer nor the describing agent changes the repository:
 
 - The `diff` stage records the tree it reviews, and the review and the description always start from it: what an interrupted call left behind is put back first, on a resumed run too, and counts as that call's change. A resumed review or description whose tree Git has since pruned is refused.
 - After each call, even a failed one, what the agent changed is put back (`.gitignore` files first, so that an ignored file is never taken for an addition), and it starts again in a new conversation, once per run.
@@ -117,7 +117,7 @@ Working fast with agents leaves waste in the code. `factory run tidy` removes it
 | `survey` | Reads what the scout needs: recent code, that is the commits of the main branch since the last cleanup merged (the last commit that brought a `.factory/issues/tidy-*/plan.md` to it), or of the last 14 days before the first one, 200 at most, merges aside, with their files and how often each changed; the files of the open pull requests; the cleanups closed without being merged |
 | `scout`, `scouted` | The `scout` agent, with the `plan` profile, explores recent code first, then the rest of the repository, compares candidates and answers with the cleanup that comes first, or none. A cleanup changes nothing observable: it removes the waste that fast changes leave (duplicates, dead code, needless defenses, functions too long to follow, departures from the conventions, stale comments and documentation, weak tests), leaves the files of the open pull requests alone and proposes no cleanup like those the operator closed. It answers with the cleanup's `issue.md` and `spec.md`: the gate reads them as `issue-save` does and refuses a dependency, since a cleanup starts from the main branch alone, otherwise the answer goes back to the scout (2 times at most). When none is worth a pull request, the run goes back to the main branch, deletes its empty branch (`release`) and ends there |
 | `commit-spec` | Commits the cleanup's `issue.md` and `spec.md` on the run's branch (`docs: spec tidy-<run id>`) |
-| `plan` … `attach` | As in `implement`, from that spec; the pull request links the spec at its own commit |
+| `plan` … `attach` | As in `implement`, from that spec, except that the review sends the change back 2 times at most; the pull request links the spec at its own commit |
 
 The pull request is your approval: merge it, or close it, and the scout does not propose that cleanup again. A defect the scout finds is named in the issue, never fixed by a cleanup. Until it is merged, a cleanup lives on its branch only, so the Kanban does not show it: follow its run under `/runs/`; once merged, it is an issue like the others, in Done. The run needs the repository on its main branch without changes, as `implement` does: run it from a clone kept for it rather than the one you work in. A cleanup is named after its run: its branch and its folder `.factory/issues/tidy-<run id>/` lead to the run under `/runs/`, and its title is that of its pull request and `issue.md`. A failed run stops on its branch and resumes as `implement` does.
 

package/dist/app/workflows/implement.js

@@ -56,7 +56,7 @@ export function implement(definition) {
             }),
             reviewed: node(judgeReview, {
                 approved: "describe",
-                changes: retry("implement", 2),
+                changes: retry("implement", 4),
                 invalid: retry("review", 2),
             }),
             describe: node(describer(profiles.describe ?? profiles.review), {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@clemrig/factory",
-  "version": "0.9.0",
+  "version": "0.10.0",
   "description": "Take a repository's issues from spec to pull request with agents, watch them on a local Kanban, and run the repository's own workflows",
   "license": "MIT",
   "repository": {

package/prompts/review.md

@@ -18,6 +18,12 @@ departures from the repository's conventions. Follow every path the change must
 concurrent requests and recoveries included, not only the one that succeeds: for each mechanism the change touches,
 such as a state machine, a queue, a lock or a cache, go through each of its states and transitions.
 
+Judge on what you established, never on what the code seems to do. A criterion is `met` when a test you ran, the
+application you tried or a code path you followed to its end shows it, not because a class name, a comment or a
+test's title suggests it. A finding states only behavior you established the same way; when the defect shows in the
+application and you can run it, reproduce it there. A risk you could not establish is not a defect: report it as
+`weak_test` when no test covers it, or leave it out.
+
 When you find a defect, look for the same defect everywhere in the change: every text that shows a count, every
 path that takes the same locks, every transition of the same mechanism. Report it once, naming each place, so that
 it is fixed everywhere at once; when several defects come from one mechanism, report the mechanism, with the
@@ -33,11 +39,13 @@ answer. Never modify, stage, commit or delete anything in the repository: the ru
 your review, restores what changed and starts the review again.
 
 When the change is large (`large` is true), or spans several applications or layers, and you can launch
-sub-agents, split the review between them, five at most: each takes one part of the change, such as an
-application, a layer or a feature, and one more takes the paths that cross the parts: failures, interruptions,
-concurrent requests and recoveries. Give each the spec, its part, the files concerned and where to read the diff,
-and tell it to report its findings with their evidence, to look for each defect's other occurrences, to change
-nothing and to launch no agent itself. Check their reports, then give one answer for the whole change.
+sub-agents, split the whole review between them, five at most, and review no part yourself: a part you keep while
+you coordinate the others gets less attention than theirs. Each takes one part of the change, such as an
+application, a layer or a feature, with its checks in a browser when it shows in a web interface; one more, always,
+takes the paths that cross the parts: failures, interruptions, concurrent requests and recoveries. Give each the
+spec, its part, the files concerned and where to read the diff, and tell it to report its findings with their
+evidence, established as above, to look for each defect's other occurrences, to change nothing and to launch no
+agent itself. Check their reports, then give one answer for the whole change.
 
 Answer with:
 
@@ -64,7 +72,10 @@ the change fixes it, put it in `resolved`, with its `summary` and the `evidence`
 keep it in `findings`, with the same nature unless the code shows that nature was wrong, which its summary then
 says. Look at how each was fixed: a fix that adds a special case where the mechanism should change, or that makes
 the change harder to follow without need, is a finding, `maintainability`, or the defect's own nature when the
-mechanism still fails elsewhere. Then answer your whole review again: every criterion, and your findings with the
-previous ones that still hold, each once.
+mechanism still fails elsewhere. Then try each fix in the situations around the one its finding described: the
+other sizes, input modes and states of what the fix changed, such as a narrow screen, a touch, a keyboard, an empty
+or long value, a failure. A fix that breaks something that worked in the previous version is a `regression`. Then
+answer your whole review again: every criterion, and your findings with the previous ones that still hold, each
+once.
 
 The change is approved only when every criterion is met and no finding is blocking.