next-issue 2.9.0 → 2.11.0

package/README.md

@@ -167,7 +167,9 @@ comment: `### Review: approved`, `### Review: changes requested` and
 `### Fixer: no change`. The account is the same for all of them, so the heading,
 not the author, tells them apart. The agents read the comments of the issue, not
 of the pull request, so a comment of an agent never returns to a prompt: the
-dispute of the fixer reaches the reviewer through the state file.
+dispute of the fixer reaches the reviewer through the state file. The one
+exception is the answer to a hand-over, see
+[Retry an issue that waits for a person](#retry-an-issue-that-waits-for-a-person).
 
 ## Stop a run
 
@@ -233,11 +235,11 @@ A short text summary of the same data goes to standard output. Use `--json` to
 get the full summary object there instead, so the harness fits in a pipe. All
 progress goes to standard error.
 
-Recorded for each step: `claim`, `comments`, `worktree`, `setup`, `implement`,
-`push`, `pull-request`, `checks`, `review` and `fix`, each with a `.start`,
-`.end` or `.error` event and a duration in `ms`. Every `git` and `gh` command
-is recorded with its exit code and duration. A `tool` event holds the tool
-name, and for a `bash` tool the command too. Each agent call adds a `usage`
+Recorded for each step: `claim`, `comments`, `reply`, `worktree`, `setup`,
+`implement`, `push`, `pull-request`, `checks`, `review` and `fix`, each with a
+`.start`, `.end` or `.error` event and a duration in `ms`. Every `git` and `gh`
+command is recorded with its exit code and duration. A `tool` event holds the
+tool name, and for a `bash` tool the command too. Each agent call adds a `usage`
 event with the tokens, the cost estimate in dollars, the turn count, the
 tool-call count, the model and the session id.
 
@@ -386,6 +388,15 @@ the pull request and the step that the last run reached, so the work carries on
 where it stopped. A commit that you pushed to the pull request in the meantime
 is fetched first. An issue that still has the label is not claimed.
 
+To give guidance, answer the hand-over with a comment in the same place: on the
+pull request, or on the issue when there is no pull request. When the run
+resumes the issue, it reads the comments posted after its last
+`next-issue needs help:` comment there. Only comments from users with write
+access to the repository count, for the hand-over comment too. The implementer
+and every fixer of that run get them in their prompt, under the heading
+`Answer of a person to the hand-over`. A short reply such as "ready, take it" is
+enough. With no such comment, the run resumes with no guidance.
+
 The `--reset` flag does the same for any issue that the run handles, whether or
 not it was handed over. It touches only an issue that the run handles, so the
 state of an issue that a label still holds back stays as it was.
@@ -450,12 +461,25 @@ to it. The worker:
   file. It works on macOS and Linux;
 - holds `next-issue.lock` in the same folder, with its process id, so only one
   worker runs. A lock whose process is no longer alive is taken over;
+- keeps the repositories it knows in `next-issue.roots.json` in the same
+  folder, so it still knows them after a restart;
 - runs at most one run per repository, and runs in several repositories at
   once. Each run is like `next-issue` without `--watch`, and the client says
   whether it merges;
 - sends every client the progress of every run when it connects and on every
   change: the issue, the step, the tool count, the last tool and the handled
   issues;
+- sends every client a `questions` message for each repository: the open
+  issues with the `needsHuman` label, each with its number, title, the reason
+  of the hand-over and the url of the comment that holds it. It reads them
+  from GitHub, so the list is the same after a restart. It sends them when a
+  client connects, for each repository it knows, after each issue of a run
+  ends, and when a client sends `{ "type": "questions", "root": "<path>" }`.
+  An issue that no longer has the label leaves the list. An issue with the
+  label but no hand-over comment is in the list with an empty reason and url.
+  The issues are read one at a time. A client never gets an older list after
+  a newer one. When the list cannot be read, the client gets
+  `{ "type": "questions-error", "root": "<path>", "message": "<text>" }`;
 - stops a run, when a client asks, like a second `Ctrl-C`: after the current
   round, with the state kept;
 - stays when all clients close;

package/dist/fix.mjs

@@ -10,7 +10,7 @@ export async function fixRound(job, reason, detail, earlier, kind, force = false
     const result = await log.step('fix', { reason }, () => ports.runAgent(log, {
         name: 'fixer',
         cwd: worktree,
-        prompt: fixPrompt(issue, reason, detail, earlier, kind),
+        prompt: fixPrompt(issue, reason, detail, earlier, kind, job.reply),
         profile: CODING,
         model: context.config.models.fixer,
         timeoutMs: context.config.agentTimeoutMinutes * 60_000,

package/dist/github.mjs

@@ -74,6 +74,41 @@ export async function issueComments(repo, issue) {
     const parsed = JSON.parse(raw);
     return parsed.comments.map((comment) => `@${comment.author.login}: ${comment.body}`);
 }
+export async function threadComments(repo, number) {
+    const raw = await repeatable([
+        'api',
+        '--paginate',
+        `repos/${slug(repo)}/issues/${number}/comments`,
+        '--jq',
+        '.[] | [.user.login, .body, .html_url] | @json',
+    ], { cwd: repo.root });
+    return raw
+        .split('\n')
+        .filter((line) => line.length > 0)
+        .map((line) => {
+        let parsed;
+        try {
+            parsed = JSON.parse(line);
+        }
+        catch {
+            throw new Error(`gh gave a comment of #${number} that is not valid JSON: ${line}`);
+        }
+        const [author, body, url] = parsed;
+        return url === undefined ? { author, body: body ?? '' } : { author, body: body ?? '', url };
+    });
+}
+export async function canWrite(repo, login) {
+    try {
+        const permission = await repeatable(['api', `repos/${slug(repo)}/collaborators/${login}/permission`, '--jq', '.permission'], { cwd: repo.root });
+        return ['admin', 'write'].includes(permission.trim());
+    }
+    catch (error) {
+        if (error instanceof Error && /HTTP 404/.test(error.message)) {
+            return false;
+        }
+        throw error;
+    }
+}
 export async function openBlockers(repo, issue) {
     const raw = await repeatable([
         'api',

package/dist/handover.mjs

@@ -1,6 +1,58 @@
 import { setStatus } from './job.mjs';
 import { message } from './observe.mjs';
-import { dropState, writeState } from './state.mjs';
+import { dropState, readState, writeState } from './state.mjs';
+export const HAND_OVER = 'next-issue needs help:';
+function writers(repo, ports) {
+    const access = new Map();
+    return (author) => {
+        if (!access.has(author)) {
+            access.set(author, ports.canWrite(repo, author));
+        }
+        return access.get(author);
+    };
+}
+async function lastHandOver(comments, writer) {
+    for (let index = comments.length - 1; index >= 0; index -= 1) {
+        const comment = comments[index];
+        if (comment.body.startsWith(HAND_OVER) && (await writer(comment.author))) {
+            return index;
+        }
+    }
+    return -1;
+}
+export async function waitingQuestions(repo, config, ports) {
+    const writer = writers(repo, ports);
+    const issues = await ports.openIssues(repo, config.issueLimit, [config.labels.needsHuman]);
+    const questions = [];
+    for (const issue of issues) {
+        const pr = (await readState(repo, issue.number))?.pr;
+        const comments = await ports.threadComments(repo, pr ?? issue.number);
+        const comment = comments[await lastHandOver(comments, writer)];
+        questions.push({
+            issue: issue.number,
+            title: issue.title,
+            reason: comment?.body.slice(HAND_OVER.length).trim() ?? '',
+            url: comment?.url ?? '',
+        });
+    }
+    return questions;
+}
+export async function handOverReply(context, issue, pr) {
+    const { repo, ports } = context;
+    const comments = await ports.threadComments(repo, pr ?? issue);
+    const writer = writers(repo, ports);
+    const last = await lastHandOver(comments, writer);
+    if (last === -1) {
+        return [];
+    }
+    const reply = [];
+    for (const comment of comments.slice(last + 1)) {
+        if (await writer(comment.author)) {
+            reply.push(`@${comment.author}: ${comment.body}`);
+        }
+    }
+    return reply;
+}
 export function escalate(job, reason) {
     return handOver(job.context, job.log, job.issue.number, job.state, reason);
 }
@@ -57,7 +109,7 @@ export async function handOver(context, log, issue, state, reason) {
     log.event('hand-over', { reason }, 'quiet');
     await writeState(context.repo, { ...state, handedOver: true });
     await setStatus(context, issue, state.pr, context.config.labels.needsHuman);
-    const body = `next-issue needs help: ${reason}`;
+    const body = `${HAND_OVER} ${reason}`;
     if (state.pr === undefined) {
         await context.ports.commentOnIssue(context.repo, issue, body);
     }

package/dist/pipeline.mjs

@@ -2,7 +2,7 @@ import { CODING } from './agents.mjs';
 import { commitWork } from './fix.mjs';
 import { checkGate } from './gate.mjs';
 import { branchName } from './git.mjs';
-import { cleanUp, escalate, finish, handOver, skipped, stop } from './handover.mjs';
+import { cleanUp, escalate, finish, handOver, handOverReply, skipped, stop } from './handover.mjs';
 import { openPr, setStatus } from './job.mjs';
 import { afterMerge, fixBase } from './merge.mjs';
 import { message } from './observe.mjs';
@@ -112,7 +112,7 @@ export async function processIssue(context, issue) {
             ? resetState(saved)
             : saved;
     try {
-        const report = await work(context, issue, log, state, saved === undefined);
+        const report = await work(context, issue, log, state, saved === undefined, saved?.handedOver === true);
         log.event('issue.end', { outcome: report.outcome, reason: report.reason, ms: report.ms }, 'quiet');
         return report;
     }
@@ -136,7 +136,7 @@ export async function processIssue(context, issue) {
         };
     }
 }
-async function work(context, issue, log, state, start) {
+async function work(context, issue, log, state, start, handedOver) {
     const started = Date.now();
     const { repo, config, ports } = context;
     const branch = state.branch;
@@ -145,6 +145,9 @@ async function work(context, issue, log, state, start) {
         await writeState(repo, state);
     });
     const comments = await log.step('comments', {}, () => ports.issueComments(repo, issue.number));
+    const reply = handedOver
+        ? await log.step('reply', {}, () => handOverReply(context, issue.number, state.pr))
+        : [];
     const added = await log.step('worktree', { branch }, () => ports.addWorktree(repo, issue.number, branch, context.base, config.remote, start));
     const worktree = added.path;
     if (!added.baseUpdated) {
@@ -158,6 +161,7 @@ async function work(context, issue, log, state, start) {
         branch,
         worktree,
         comments,
+        reply,
         started,
         noChecks: false,
         mainBroken: false,
@@ -179,7 +183,7 @@ async function work(context, issue, log, state, start) {
         const result = await log.step('implement', {}, () => ports.runAgent(log, {
             name: 'implementer',
             cwd: worktree,
-            prompt: implementPrompt(issue, comments),
+            prompt: implementPrompt(issue, comments, reply),
             profile: CODING,
             model: config.models.implementer,
             timeoutMs: config.agentTimeoutMinutes * 60_000,

package/dist/ports.mjs

@@ -1,10 +1,11 @@
 import { runAgent } from './agents.mjs';
 import { run } from './exec.mjs';
 import { addWorktree, commitAll, deleteBranch, diff, discardChanges, disputeBranches, findWorktree, isDirty, latestSubject, push, removeWorktree, resetToBase, revision, setAside, } from './git.mjs';
-import { commentOnIssue, commentOnPr, createPr, failedBaseLogs, failedCheckLogs, findPr, issueComments, markPrReady, mergeCommit, mergePr, openBlockers, openIssues, prConflicts, prMerged, reopenIssue, setLabel, viewIssue, waitForBaseChecks, waitForChecks, } from './github.mjs';
+import { canWrite, commentOnIssue, commentOnPr, createPr, failedBaseLogs, failedCheckLogs, findPr, issueComments, markPrReady, mergeCommit, mergePr, openBlockers, openIssues, prConflicts, prMerged, reopenIssue, setLabel, threadComments, viewIssue, waitForBaseChecks, waitForChecks, } from './github.mjs';
 import { detectSetupCommand } from './setup.mjs';
 export const PORTS = {
     addWorktree,
+    canWrite,
     commentOnIssue,
     commentOnPr,
     commitAll,
@@ -37,6 +38,7 @@ export const PORTS = {
     runSetup: async (command, cwd, timeoutMs) => run('bash', ['-lc', command], { cwd, timeoutMs }),
     setAside,
     setLabel,
+    threadComments,
     viewIssue,
     waitForBaseChecks,
     waitForChecks,

package/dist/prompts.mjs

@@ -11,15 +11,27 @@ function issueBlock(issue, comments) {
         .filter((part) => part.length > 0)
         .join('\n\n');
 }
-export function implementPrompt(issue, comments) {
+function replyBlock(reply) {
+    return reply.length > 0
+        ? [
+            '## Answer of a person to the hand-over',
+            'The last run handed the issue to a person. A person with write access to the repository answered as below. Follow this guidance.',
+            reply.join('\n\n'),
+        ].join('\n\n')
+        : '';
+}
+export function implementPrompt(issue, comments, reply = []) {
     return [
         'You are the implementer. Solve the issue below in this worktree.',
         issueBlock(issue, comments),
+        replyBlock(reply),
         '## Rules',
         'Read the repository conventions first. Keep the change minimal and focused on the issue.',
         'Add or update tests when the repository has a test suite.',
         COMMIT_RULES,
-    ].join('\n\n');
+    ]
+        .filter((part) => part.length > 0)
+        .join('\n\n');
 }
 export function hookPrompt(issue, comments, role, output) {
     return [
@@ -75,7 +87,7 @@ const DISPUTE_RULE = [
     'If only some of the findings are wrong, fix the correct ones and change nothing for the wrong ones. Write no `unrelated` line in that round. The next round holds only the findings that are left, and you can dispute them there.',
     'The reviewer reads that line in the next round. Do not make a change that you hold to be wrong only to end the round.',
 ].join('\n');
-export function fixPrompt(issue, reason, detail, earlier, kind) {
+export function fixPrompt(issue, reason, detail, earlier, kind, reply = []) {
     return [
         `You are the fixer. Correct the work for issue #${issue.number}: ${issue.title}.`,
         `## Reason\n${reason}`,
@@ -87,6 +99,7 @@ export function fixPrompt(issue, reason, detail, earlier, kind) {
                 'Keep those earlier fixes. Do not undo work that a later round did not question.',
             ].join('\n')
             : '',
+        replyBlock(reply),
         '## Rules',
         'Fix the cause, not the symptom. Keep the change minimal.',
         kind === 'checks' ? UNRELATED_RULE : DISPUTE_RULE,

package/dist/serve.mjs

@@ -5,16 +5,21 @@ import { join } from 'node:path';
 import { getAgentDir } from '@earendil-works/pi-coding-agent';
 import { WebSocketServer } from 'ws';
 import { parseArgs } from './args.mjs';
+import { loadConfig } from './config.mjs';
 import { stopChildren } from './exec.mjs';
+import { detectRepo } from './git.mjs';
+import { waitingQuestions } from './handover.mjs';
 import { message } from './observe.mjs';
 import { applyEvent, startProgress } from './progress.mjs';
-import { isNewer } from './protocol.mjs';
+import { PORTS } from './ports.mjs';
+import { isNewer, } from './protocol.mjs';
 import { runRepo } from './run.mjs';
 import { packageVersion } from './version.mjs';
 export function socketPaths(agentDir = getAgentDir()) {
     return {
         socketPath: join(agentDir, 'next-issue.sock'),
         lockPath: join(agentDir, 'next-issue.lock'),
+        rootsPath: join(agentDir, 'next-issue.roots.json'),
     };
 }
 function alive(pid) {
@@ -55,12 +60,39 @@ async function lock(path) {
     }
     return false;
 }
+async function readRoots(path) {
+    try {
+        const value = JSON.parse(await readFile(path, 'utf8'));
+        return new Set(Array.isArray(value) ? value.filter((root) => typeof root === 'string') : []);
+    }
+    catch {
+        return new Set();
+    }
+}
+export async function listQuestions(root) {
+    const config = await loadConfig(root);
+    return waitingQuestions(await detectRepo(root, config.remote), config, PORTS);
+}
 export async function startWorker(options) {
     if (!(await lock(options.lockPath))) {
         return undefined;
     }
     await rm(options.socketPath, { force: true });
     const runs = new Map();
+    const roots = await readRoots(options.rootsPath);
+    let saving = Promise.resolve();
+    const remember = (root) => {
+        if (roots.has(root)) {
+            return;
+        }
+        roots.add(root);
+        const value = `${JSON.stringify([...roots], null, 2)}\n`;
+        saving = saving
+            .then(() => writeFile(options.rootsPath, value, { mode: 0o600 }))
+            .catch(() => undefined);
+    };
+    const reads = new Map();
+    const delivered = new WeakMap();
     let restart = false;
     let finish = () => undefined;
     const closed = new Promise((resolve) => {
@@ -81,6 +113,26 @@ export async function startWorker(options) {
             send(socket, value);
         }
     };
+    const sendQuestions = async (root, targets) => {
+        remember(root);
+        const read = (reads.get(root) ?? 0) + 1;
+        reads.set(root, read);
+        let value;
+        try {
+            value = { type: 'questions', root, questions: await options.questions(root) };
+        }
+        catch (error) {
+            value = { type: 'questions-error', root, message: message(error) };
+        }
+        for (const socket of targets()) {
+            const seen = delivered.get(socket) ?? new Map();
+            if ((seen.get(root) ?? 0) < read) {
+                seen.set(root, read);
+                delivered.set(socket, seen);
+                send(socket, value);
+            }
+        }
+    };
     const busy = () => [...runs.values()].some((run) => run.progress.status === 'running' || run.progress.status === 'stopping');
     const close = async () => {
         for (const socket of sockets.clients) {
@@ -89,6 +141,7 @@ export async function startWorker(options) {
         await new Promise((resolve) => sockets.close(() => resolve()));
         await new Promise((resolve) => server.close(() => resolve()));
         await rm(options.socketPath, { force: true });
+        await saving;
         await rm(options.lockPath, { force: true });
         finish();
     };
@@ -109,6 +162,7 @@ export async function startWorker(options) {
         }
         const run = { progress: startProgress(), stop: false };
         runs.set(root, run);
+        remember(root);
         const interrupts = {
             afterIssue: () => run.stop,
             afterRound: () => run.stop,
@@ -132,7 +186,12 @@ export async function startWorker(options) {
         options
             .runRepo(root, { ...parseArgs([]), merge }, {
             interrupts,
-            onEvent: (record) => update(applyEvent(run.progress, record)),
+            onEvent: (record) => {
+                update(applyEvent(run.progress, record));
+                if (record['kind'] === 'issue.end' || record['kind'] === 'issue.error') {
+                    void sendQuestions(root, () => sockets.clients);
+                }
+            },
         })
             .then(() => {
             const status = run.stop
@@ -157,6 +216,9 @@ export async function startWorker(options) {
     sockets.on('connection', (socket) => {
         send(socket, { type: 'hello', version: options.version });
         send(socket, snapshot());
+        for (const root of roots) {
+            void sendQuestions(root, () => [socket]);
+        }
         socket.on('message', (data) => {
             let value;
             try {
@@ -177,6 +239,9 @@ export async function startWorker(options) {
             else if (value.type === 'stop') {
                 stop(value.root);
             }
+            else if (value.type === 'questions') {
+                void sendQuestions(value.root, () => [socket]);
+            }
         });
     });
     await new Promise((resolve, reject) => {
@@ -200,6 +265,7 @@ export async function serve() {
         ...paths,
         version: await packageVersion(),
         runRepo,
+        questions: listQuestions,
     });
     if (worker === undefined) {
         process.stderr.write('next-issue: another worker is running\n');

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "next-issue",
-  "version": "2.9.0",
+  "version": "2.11.0",
   "description": "Autonomous issue harness driven by the pi SDK",
   "keywords": [
     "github",