@ravshansbox/pi-next-issue 1.2.0 → 2.3.1

package/README.md

@@ -1,53 +1,29 @@
 # pi-next-issue
 
-A [pi](https://github.com/earendil-works/pi) extension that works through the open GitHub issues of a repository in the background, while you keep using pi.
+A [pi](https://github.com/earendil-works/pi) extension that works through the open GitHub issues of a repository in the background, while you keep using pi. The work is done by the [`next-issue`](../next-issue) worker, which this extension starts and talks to.
 
 ```bash
 pi install npm:@ravshansbox/pi-next-issue
 ```
 
-It needs `git` and an authenticated [`gh`](https://cli.github.com/). It works with github.com and GitHub Enterprise remotes.
+It needs `git`, `npx` and an authenticated [`gh`](https://cli.github.com/). It works on macOS and Linux.
 
 ## Use
 
 Run this in pi, inside a git repository:
 
-- `/next-issue` starts a run when none is going, and stops the run when one is going or stopping. A stopped issue keeps its saved state, and the next run resumes it.
+- `/next-issue` starts a run of this repository when none is going, and stops the run when one is going. A stop takes effect after the current round, and the issue keeps its saved state, so the next run resumes it.
 
-The footer shows the issue, the stage, the number of tool calls and the time spent on the issue, for example `next-issue #12 · review 1/3 · 37 tools · 4m12s`. After a run ends, it keeps the last summary. A line above the editor shows the last tool call. In the terminal, both use the dim colour of the theme, like the rest of the footer. A note appears when an issue is merged or needs a human.
+The footer shows the issue, the stage, the number of tool calls and the time spent on the issue, for example `next-issue #12 · review 1 · 37 tools · 4m12s`. After a run ends, it keeps the last summary. A line above the editor shows the last tool call. In the terminal, both use the dim colour of the theme, like the rest of the footer. A note appears when an issue is done, needs a human or fails.
 
-The run does not belong to a session: it goes on after `/new`, `/resume`, a fork or a reload, and the new session shows it. It stops when you quit pi.
+## Worker
 
-## What it does
+The extension does no work itself. On `/next-issue` it connects to the `next-issue` worker on the socket file `~/.pi/agent/next-issue.sock`. When no worker answers, it starts one with `npx -y next-issue@latest serve`, in the background, and writes the output of the worker to `~/.pi/agent/next-issue-worker.log`.
 
-Each loop fetches the 200 oldest open issues with the `status:todo` label and none of `status:in-progress`, `status:done`, `status:needs-human` or `status:blocked`, and the open issues with saved state that it can read. It takes the oldest one with saved state and none of `status:done`, `status:needs-human` or `status:blocked`, else the oldest one with the `status:todo` label and none of `status:in-progress`, `status:done`, `status:needs-human` or `status:blocked`. The run ends when no such issue is left, or at the first error. For each issue:
+- Every pi on the machine shares the one worker, and every pi in a repository shows the run of that repository, whoever started it.
+- Opening pi does not start a worker. It only shows a run when a worker is going.
+- The run does not belong to pi: it goes on after `/new`, `/resume`, a fork, a reload or quitting pi.
+- The worker stays when every pi is closed. Stop it with `kill $(cat ~/.pi/agent/next-issue.lock)`.
+- When the extension is newer than the worker, the worker closes once no run is going, and the extension starts the newer worker.
 
-1. Swap `status:todo` for `status:in-progress`.
-2. Add a git worktree at `.worktrees/<n>` on the `<n>` branch, from the remote default branch.
-3. The implementer agent does the work. The extension commits it, with the subject from a `commit:` line in the reply of the agent, and pushes. When a resumed issue already has a commit ahead of the base, the extension keeps it and skips the implementer.
-4. Open a pull request that closes the issue.
-5. Wait for the checks. On a red build, the fixer agent gets the failed logs. It gets at most `maxCiFixes` such fixes for the whole issue, and this count includes the checks after review fixes.
-6. The reviewer agent judges the diff and gives a verdict with `blocking` and `minor` findings, which goes on the pull request. Any change the reviewer makes is thrown away.
-7. While review rounds are left, the fixer gets every finding, and the checks and the review run again. In the last round only a blocking finding stops the merge.
-8. Squash-merge the pull request, wait until it is merged, set `status:done`, and remove the worktree, the local branch, the remote branch and the saved state.
-
-An issue that cannot go on gets `status:needs-human` and a comment with the reason. Its worktree stays for you to look at.
-
-The implementer and the reviewer read the issue title, the body and the comments. They get only comments by the owner, members and collaborators of the repository. The body is passed as it is, and its author can edit it after you add the label, so check the body before you add the label.
-
-Each agent is a pi session in the worktree. It uses the model and thinking level from your pi settings, and loads all of your extensions except this one.
-
-The state of each issue is saved in `.worktrees/<n>.json`. Each loop removes the saved state of an issue that is closed. A run holds `.worktrees/run.lock`, which keeps the process id, so that only one pi at a time works on a repository. A second run fails at once. A lock whose process is no longer alive is taken over. If the operating system gives that process id to another process, the lock stays; remove `.worktrees/run.lock` by hand when no run is going. The extension adds `/.worktrees/` to `.git/info/exclude`.
-
-## Config
-
-An optional `.pi/next-issue.json` in the repository sets these fields:
-
-| Field                 | Default    | Meaning                                                                           |
-| --------------------- | ---------- | --------------------------------------------------------------------------------- |
-| `remote`              | `"origin"` | The git remote to push to.                                                        |
-| `maxCiFixes`          | `3`        | Fixes of red checks for the whole issue, including the checks after review fixes. |
-| `maxReviewRounds`     | `3`        | Review rounds per issue.                                                          |
-| `checkTimeoutMinutes` | `30`       | The longest wait for the checks, and for a merge.                                 |
-| `diffMaxChars`        | `60000`    | The longest diff that the reviewer gets.                                          |
-| `logMaxChars`         | `20000`    | The longest failed log that the fixer gets.                                       |
+Each run asks the worker to merge the pull requests. See the [`next-issue` README](../next-issue/README.md) for what a run does, its labels, its state and its config.

package/index.ts

@@ -1,19 +1,24 @@
-import type { ExtensionAPI, ExtensionContext } from '@earendil-works/pi-coding-agent';
-import { loadConfig } from './src/config.ts';
+import { readFileSync } from 'node:fs';
+import { join } from 'node:path';
+import {
+  type ExtensionAPI,
+  type ExtensionContext,
+  getAgentDir,
+} from '@earendil-works/pi-coding-agent';
 import { repoRoot } from './src/git.ts';
-import { createPorts } from './src/ports.ts';
-import { type Progress, Runner } from './src/runner.ts';
+import { type Progress, spawnWorker, WorkerClient, type WorkerEvent } from './src/worker.ts';
 
-type Launch = (cwd: string) => Promise<Runner>;
-
-type Holder = { runner?: Runner; launching?: boolean };
+export type Options = {
+  client: () => WorkerClient;
+  root: (cwd: string) => Promise<string>;
+};
 
 const KEY = Symbol.for('pi-next-issue');
 const STATUS_KEY = 'next-issue';
 
-function holder(): Holder {
-  const store = globalThis as Record<symbol, Holder | undefined>;
-  store[KEY] ??= {};
+function shared(create: () => WorkerClient): WorkerClient {
+  const store = globalThis as Record<symbol, WorkerClient | undefined>;
+  store[KEY] ??= create();
   return store[KEY];
 }
 
@@ -24,19 +29,23 @@ function duration(ms: number): string {
 }
 
 function outcomes(progress: Progress): string {
-  const merged = progress.handled.filter((item) => item.outcome === 'merged').length;
-  const human = progress.handled.length - merged;
-  const parts = [`${merged} merged`];
+  const done = progress.handled.filter((item) => item.outcome === 'done').length;
+  const human = progress.handled.filter((item) => item.outcome === 'needs-human').length;
+  const failed = progress.handled.filter((item) => item.outcome === 'error').length;
+  const parts = [`${done} done`];
   if (human > 0) {
     parts.push(`${human} needs a human`);
   }
+  if (failed > 0) {
+    parts.push(`${failed} failed`);
+  }
   return parts.join(', ');
 }
 
+const OUTCOME_NOTES = { done: 'done', 'needs-human': 'needs a human', error: 'failed' } as const;
+
 export function formatStatus(progress: Progress, now: number): string {
   switch (progress.status) {
-    case 'idle':
-      return 'next-issue · starting';
     case 'running': {
       if (progress.issue === undefined) {
         return 'next-issue · looking for issues';
@@ -59,39 +68,58 @@ export function formatStatus(progress: Progress, now: number): string {
   }
 }
 
-const launchFromRepo: Launch = async (cwd) => {
-  const root = await repoRoot(cwd);
-  const config = await loadConfig(root);
-  return new Runner(await createPorts(root, config), config);
+function packageVersion(): string {
+  const raw = readFileSync(join(import.meta.dirname, 'package.json'), 'utf8');
+  return (JSON.parse(raw) as { version: string }).version;
+}
+
+const fromMachine: Options = {
+  client: () => {
+    const agentDir = getAgentDir();
+    return new WorkerClient({
+      socketPath: join(agentDir, 'next-issue.sock'),
+      version: packageVersion(),
+      spawnWorker: () => spawnWorker(agentDir),
+    });
+  },
+  root: repoRoot,
 };
 
-export function createExtension(launch: Launch) {
+export function createExtension(options: Options) {
   return (pi: ExtensionAPI) => {
     let ui: ExtensionContext['ui'] | undefined;
     let coloured = false;
+    let root: string | undefined;
     let timer: ReturnType<typeof setInterval> | undefined;
     let unsubscribe: (() => void) | undefined;
     let handled = 0;
     let status: Progress['status'] | undefined;
 
+    const current = (): Progress | undefined =>
+      root === undefined ? undefined : shared(options.client).runs[root];
+
     const render = () => {
-      const runner = holder().runner;
-      if (ui === undefined || runner === undefined) {
+      const progress = current();
+      if (ui === undefined || progress === undefined) {
         return;
       }
-      const progress = runner.progress;
-      const theme = ui.theme;
-      const dim = (text: string) => (coloured ? theme.fg('dim', text) : text);
+      const dim = (text: string) => (coloured ? ui!.theme.fg('dim', text) : text);
       ui.setStatus(STATUS_KEY, dim(formatStatus(progress, Date.now())));
       const active = progress.status === 'running' && progress.lastTool !== undefined;
       ui.setWidget(STATUS_KEY, active ? [dim(`↳ ${progress.lastTool}`)] : undefined);
     };
 
-    const onChange = (progress: Progress) => {
+    const onProgress = (progress: Progress | undefined) => {
+      if (progress === undefined) {
+        return;
+      }
+      if (progress.handled.length < handled) {
+        handled = 0;
+      }
       for (const item of progress.handled.slice(handled)) {
         ui?.notify(
-          `next-issue: #${item.issue} ${item.outcome === 'merged' ? 'merged' : 'needs a human'}`,
-          item.outcome === 'merged' ? 'info' : 'warning',
+          `next-issue: #${item.issue} ${OUTCOME_NOTES[item.outcome]}`,
+          item.outcome === 'done' ? 'info' : 'warning',
         );
       }
       handled = progress.handled.length;
@@ -108,82 +136,79 @@ export function createExtension(launch: Launch) {
       render();
     };
 
-    const follow = (runner: Runner) => {
+    const onEvent = (event: WorkerEvent) => {
+      if (event.type === 'runs') {
+        onProgress(root === undefined ? undefined : event.runs[root]);
+      } else if (event.root === root) {
+        ui?.notify(`next-issue cannot start: ${event.message}`, 'error');
+      }
+    };
+
+    const follow = () => {
       unsubscribe?.();
-      handled = runner.progress.handled.length;
-      status = runner.progress.status;
-      unsubscribe = runner.subscribe(onChange);
+      const progress = current();
+      handled = progress?.handled.length ?? 0;
+      status = progress?.status;
+      unsubscribe = shared(options.client).subscribe(onEvent);
       render();
     };
 
-    const detach = () => {
-      unsubscribe?.();
-      unsubscribe = undefined;
-      if (timer !== undefined) {
-        clearInterval(timer);
-        timer = undefined;
-      }
-      ui = undefined;
+    const show = (ctx: ExtensionContext) => {
+      ui = ctx.ui;
+      coloured = ctx.mode === 'tui';
+      timer ??= setInterval(render, 1000);
+      timer.unref?.();
     };
 
     pi.on('session_start', async (_event, ctx) => {
       if (!ctx.hasUI) {
         return;
       }
-      ui = ctx.ui;
-      coloured = ctx.mode === 'tui';
-      timer ??= setInterval(render, 1000);
-      timer.unref?.();
-      const runner = holder().runner;
-      if (runner !== undefined) {
-        follow(runner);
+      show(ctx);
+      root = await options.root(ctx.cwd).catch(() => undefined);
+      if (root === undefined) {
+        return;
       }
+      follow();
+      await shared(options.client).attach();
     });
 
-    pi.on('session_shutdown', async (event) => {
-      detach();
-      if (event.reason === 'quit') {
-        holder().runner?.stop();
+    pi.on('session_shutdown', async () => {
+      unsubscribe?.();
+      unsubscribe = undefined;
+      if (timer !== undefined) {
+        clearInterval(timer);
+        timer = undefined;
       }
+      ui = undefined;
     });
 
     pi.registerCommand('next-issue', {
       description: 'Start or stop working through the GitHub issues of this repository',
       handler: async (_args, ctx) => {
-        const current = holder().runner;
-        const busy =
-          current?.progress.status === 'running' || current?.progress.status === 'stopping';
-        if (busy) {
-          current?.stop();
-          return;
-        }
-        if (holder().launching) {
-          ctx.ui.notify('next-issue is starting already', 'warning');
-          return;
-        }
-        holder().launching = true;
-        let runner: Runner;
         try {
-          runner = await launch(ctx.cwd);
+          root = await options.root(ctx.cwd);
+          if (ctx.hasUI) {
+            show(ctx);
+            follow();
+          }
+          const client = shared(options.client);
+          await client.ensure();
+          const progress = client.runs[root];
+          if (progress?.status === 'running' || progress?.status === 'stopping') {
+            client.send({ type: 'stop', root });
+          } else {
+            client.send({ type: 'start', root, merge: true });
+          }
         } catch (error) {
           ctx.ui.notify(
             `next-issue cannot start: ${error instanceof Error ? error.message : String(error)}`,
             'error',
           );
-          return;
-        } finally {
-          holder().launching = false;
-        }
-        holder().runner = runner;
-        if (ctx.hasUI) {
-          ui = ctx.ui;
-          coloured = ctx.mode === 'tui';
-          follow(runner);
         }
-        void runner.run();
       },
     });
   };
 }
 
-export default createExtension(launchFromRepo);
+export default createExtension(fromMachine);

package/package.json

@@ -1,16 +1,17 @@
 {
   "name": "@ravshansbox/pi-next-issue",
-  "version": "1.2.0",
-  "description": "Pi extension that works through GitHub issues in the background",
+  "version": "2.3.1",
+  "description": "Pi extension that works through GitHub issues with the next-issue worker",
   "license": "MIT",
   "type": "module",
   "repository": {
     "type": "git",
-    "url": "git+https://github.com/ravshansbox/pi-next-issue.git"
+    "url": "git+https://github.com/ravshansbox/next-issue.git",
+    "directory": "packages/pi-next-issue"
   },
-  "homepage": "https://github.com/ravshansbox/pi-next-issue#readme",
+  "homepage": "https://github.com/ravshansbox/next-issue/tree/main/packages/pi-next-issue#readme",
   "bugs": {
-    "url": "https://github.com/ravshansbox/pi-next-issue/issues"
+    "url": "https://github.com/ravshansbox/next-issue/issues"
   },
   "keywords": [
     "pi-package",
@@ -33,11 +34,11 @@
     "check": "npm run typecheck && npm run lint && npm run format:check && npm test"
   },
   "peerDependencies": {
-    "@earendil-works/pi-coding-agent": "*",
-    "typebox": "*"
+    "@earendil-works/pi-coding-agent": "*"
   },
   "devDependencies": {
     "@types/node": "^26.6.5",
+    "@types/ws": "^8.18.2",
     "oxfmt": "^0.74.0",
     "oxlint": "^1.89.0",
     "typescript": "^7.0.2"
@@ -48,5 +49,8 @@
   ],
   "publishConfig": {
     "access": "public"
+  },
+  "dependencies": {
+    "ws": "^8.22.1"
   }
 }

package/src/git.ts

@@ -1,126 +1,5 @@
-import { appendFile, readFile } from 'node:fs/promises';
-import { isAbsolute, join } from 'node:path';
-import { must, run } from './exec.ts';
-
-export const WORKTREE_DIR = '.worktrees';
+import { must } from './exec.ts';
 
 export async function repoRoot(cwd: string): Promise<string> {
   return must('git', ['rev-parse', '--show-toplevel'], { cwd });
 }
-
-export function parseRemote(url: string): string {
-  const remote = url.trim();
-  const hosted =
-    /^(?:[\w+.-]+:\/\/)?(?:[^@/]+@)?([^/:]+)(?::\d+)?[:/]([^/]+)\/([^/]+?)(?:\.git)?\/?$/.exec(
-      remote,
-    );
-  if (hosted !== null) {
-    const [, host, owner, name] = hosted;
-    return host?.toLowerCase() === 'github.com' ? `${owner}/${name}` : `${host}/${owner}/${name}`;
-  }
-  const path = /[/:]([^/:]+)\/([^/]+?)(?:\.git)?\/?$/.exec(remote);
-  if (path === null) {
-    throw new Error(`cannot read the owner and name from the git remote ${url}`);
-  }
-  return `${path[1]}/${path[2]}`;
-}
-
-export async function remoteSlug(root: string, remote: string): Promise<string> {
-  return parseRemote(await must('git', ['remote', 'get-url', remote], { cwd: root }));
-}
-
-async function ignoreWorktrees(root: string): Promise<void> {
-  const common = await must('git', ['rev-parse', '--git-common-dir'], { cwd: root });
-  const file = join(isAbsolute(common) ? common : join(root, common), 'info', 'exclude');
-  const current = await readFile(file, 'utf8').catch(() => '');
-  if (!current.split('\n').some((line) => line.trim() === `/${WORKTREE_DIR}/`)) {
-    const gap = current.length === 0 || current.endsWith('\n') ? '' : '\n';
-    await appendFile(file, `${gap}/${WORKTREE_DIR}/\n`);
-  }
-}
-
-async function findWorktree(root: string, branch: string): Promise<string | undefined> {
-  const list = await must('git', ['worktree', 'list', '--porcelain'], { cwd: root });
-  const entry = list
-    .split('\n\n')
-    .map((block) => block.split('\n'))
-    .find((lines) => lines.includes(`branch refs/heads/${branch}`));
-  return entry?.find((line) => line.startsWith('worktree '))?.slice('worktree '.length);
-}
-
-async function hasRef(root: string, ref: string): Promise<boolean> {
-  return (await run('git', ['rev-parse', '--verify', '--quiet', ref], { cwd: root })).code === 0;
-}
-
-export async function worktree(
-  root: string,
-  remote: string,
-  base: string,
-  issue: number,
-  branch: string,
-): Promise<string> {
-  await ignoreWorktrees(root);
-  await must('git', ['fetch', remote], { cwd: root });
-  await must('git', ['worktree', 'prune'], { cwd: root });
-  const existing = await findWorktree(root, branch);
-  if (existing !== undefined) {
-    return existing;
-  }
-  const path = join(root, WORKTREE_DIR, String(issue));
-  if (await hasRef(root, `refs/heads/${branch}`)) {
-    await must('git', ['worktree', 'add', path, branch], { cwd: root });
-  } else {
-    const upstream = `${remote}/${branch}`;
-    const start = (await hasRef(root, `refs/remotes/${upstream}`)) ? upstream : `${remote}/${base}`;
-    await must('git', ['worktree', 'add', '--no-track', '-b', branch, path, start], { cwd: root });
-  }
-  return path;
-}
-
-export async function commit(cwd: string, message: string): Promise<boolean> {
-  if ((await must('git', ['status', '--porcelain'], { cwd })).length === 0) {
-    return false;
-  }
-  await must('git', ['add', '-A'], { cwd });
-  await must('git', ['commit', '-m', message], { cwd });
-  return true;
-}
-
-export async function ahead(
-  cwd: string,
-  remote: string,
-  base: string,
-): Promise<string | undefined> {
-  const range = `${remote}/${base}..HEAD`;
-  const subjects = await must('git', ['log', '--reverse', '--format=%s', range], { cwd });
-  return subjects.split('\n')[0] || undefined;
-}
-
-export async function push(cwd: string, remote: string, branch: string): Promise<void> {
-  await must('git', ['push', '-u', remote, `HEAD:refs/heads/${branch}`], { cwd });
-}
-
-export async function discard(cwd: string, remote: string, branch: string): Promise<void> {
-  await must('git', ['reset', '--hard', `${remote}/${branch}`], { cwd });
-  await must('git', ['clean', '-fd'], { cwd });
-}
-
-export async function diff(cwd: string, remote: string, base: string): Promise<string> {
-  await must('git', ['fetch', remote, base], { cwd });
-  return must('git', ['diff', `${remote}/${base}...HEAD`], { cwd });
-}
-
-export async function head(cwd: string): Promise<string> {
-  return must('git', ['rev-parse', 'HEAD'], { cwd });
-}
-
-export async function cleanup(
-  root: string,
-  remote: string,
-  branch: string,
-  path: string,
-): Promise<void> {
-  await run('git', ['worktree', 'remove', '--force', path], { cwd: root });
-  await run('git', ['branch', '-D', branch], { cwd: root });
-  await run('git', ['push', remote, '--delete', branch], { cwd: root });
-}