@sebastianandreasson/pi-autonomous-agents 0.1.0

package/README.md

@@ -0,0 +1,102 @@
+# PI Harness
+
+`pi-harness` is a portable CLI/workflow package for running a local PI-based unattended loop with:
+
+- a `developer` pass
+- a fast verification step
+- a skeptical `tester` pass
+- optional periodic multimodal visual review
+- harness-owned git finalization
+
+The package is intentionally generic. It does not know how to navigate or test a specific app on its own.
+
+## What Belongs In The Package
+
+- supervisor/orchestration
+- PI adapter/runtime integration
+- config loading
+- telemetry
+- loop guards, timeout guards, and retries
+- tester feedback + visual feedback handoff
+- harness-owned git finalize step
+- multimodal visual review client
+
+## What Stays Per Project
+
+- `TODOS.md`
+- project instructions
+- browser tests
+- visual capture flow
+- app-specific verification commands
+- app/server startup scripts
+
+## Layout
+
+```text
+packages/pi-harness/
+  package.json
+  pi.config.json
+  templates/DEVELOPER.md
+  templates/TESTER.md
+  docs/PI_SUPERVISOR.md
+  src/
+    cli.mjs
+    pi-client.mjs
+    pi-config.mjs
+    pi-prompts.mjs
+    pi-repo.mjs
+    pi-report.mjs
+    pi-rpc-adapter.mjs
+    pi-supervisor.mjs
+    pi-telemetry.mjs
+    pi-visual-once.mjs
+    pi-visual-review.mjs
+```
+
+## CLI
+
+```bash
+pi-harness once
+pi-harness run
+pi-harness report
+pi-harness visual-once
+pi-harness adapter
+pi-harness visual-review-worker
+```
+
+Use `PI_CONFIG_FILE` to point the harness at a project-local config file. If you do not provide one, the bundled generic `pi.config.json` is used as a fallback.
+
+## Setup In Another Repo
+
+After installing the package:
+
+```bash
+npm install -D @sebastianandreasson/pi-autonomous-agents
+```
+
+you can tell another agent in that repo:
+
+```text
+Find SETUP.md in @sebastianandreasson/pi-autonomous-agents and set everything up for this repository.
+```
+
+The package ships a top-level [SETUP.md](./SETUP.md) specifically for that workflow.
+
+## Generic Contracts
+
+- `taskFile`: usually `TODOS.md`
+- `developerInstructionsFile`: per-project developer instructions
+- `testerInstructionsFile`: per-project tester instructions
+- `roleModels`: optional per-role model overrides
+- `testCommand`: fast verification command
+- `visualCaptureCommand`: project-defined screenshot capture command
+- `visualFeedbackFile`: latest visual-review handoff
+- `testerFeedbackFile`: latest tester-review handoff
+
+For unattended loops, keep `testCommand` fast and bounded, such as a smoke suite. Long real-time Playwright happy-path specs belong in an explicit nightly or post-run lane, not the default developer/tester inner loop.
+
+The adapter heartbeat is PI-RPC-event based. Streaming shell output does not count as progress on its own, so long-running tools should rely on the tool-aware watchdog thresholds rather than terminal streaming.
+
+`piModel` remains the default text model, but you can override specific roles with `roleModels` such as `developer`, `developerRetry`, `developerFix`, `tester`, `testerCommit`, and `visualReview`.
+
+The harness expects screenshot capture to produce a `manifest.json` plus image files under the configured visual capture directory.

package/SETUP.md

@@ -0,0 +1,171 @@
+# Setup Guide
+
+This file is written for an autonomous coding agent that has been asked to set up `@sebastianandreasson/pi-autonomous-agents` in the current repository.
+
+## Goal
+
+Install and configure the package so this repo can run:
+
+- `pi-harness once`
+- `pi-harness run`
+- `pi-harness report`
+- `pi-harness visual-once`
+
+Do not stop after installing the dependency. Complete the repo-local setup as well.
+
+## Required End State
+
+When you are done, the consuming repo should contain:
+
+- `pi.config.json`
+- `TODOS.md` if the repo does not already have one
+- `pi/DEVELOPER.md`
+- `pi/TESTER.md`
+- package scripts for the harness commands
+- `.gitignore` entries for harness runtime/output files
+
+If the repo already has equivalent files, update them instead of creating duplicates.
+
+## Setup Steps
+
+1. Install the package.
+
+Preferred command:
+
+```bash
+npm install -D @sebastianandreasson/pi-autonomous-agents
+```
+
+If the repo uses another package manager already, use the repo-native equivalent instead.
+
+2. Create `pi.config.json`.
+
+- Start from `node_modules/@sebastianandreasson/pi-autonomous-agents/templates/pi.config.example.json`.
+- Copy it into the repo root as `pi.config.json`.
+- Update it for this repo:
+  - `taskFile`: usually `TODOS.md`
+  - `developerInstructionsFile`: `pi/DEVELOPER.md`
+  - `testerInstructionsFile`: `pi/TESTER.md`
+  - `testCommand`: a fast bounded verification command for this repo
+  - `visualCaptureCommand`: only if this repo has a real screenshot capture flow
+  - `models` / `piModel` / `visualReviewModel` / `roleModels`: configure the models actually available in this environment
+
+3. Create role instruction files.
+
+- Copy `node_modules/@sebastianandreasson/pi-autonomous-agents/templates/DEVELOPER.md` to `pi/DEVELOPER.md`.
+- Copy `node_modules/@sebastianandreasson/pi-autonomous-agents/templates/TESTER.md` to `pi/TESTER.md`.
+- Customize both files for the repo:
+  - name the actual product/app
+  - describe the real verification expectations
+  - mention project-specific constraints, startup flow, or directories
+  - keep the harness workflow intact
+
+4. Ensure `TODOS.md` exists.
+
+- If the repo already uses a task file, keep it.
+- Otherwise create a minimal `TODOS.md` with at least one phase heading and one unchecked actionable checkbox.
+
+Minimal example:
+
+```md
+## Phase 1
+
+- [ ] Define the first real task for this repo
+```
+
+5. Add package scripts.
+
+Add these scripts to the consuming repo `package.json`, adapting only if necessary:
+
+```json
+{
+  "scripts": {
+    "pi:mock": "PI_CONFIG_FILE=pi.config.json PI_TRANSPORT=mock PI_TEST_CMD= pi-harness once",
+    "pi:once": "PI_CONFIG_FILE=pi.config.json pi-harness once",
+    "pi:run": "PI_CONFIG_FILE=pi.config.json pi-harness run",
+    "pi:report": "PI_CONFIG_FILE=pi.config.json pi-harness report",
+    "pi:visual:once": "PI_CONFIG_FILE=pi.config.json pi-harness visual-once"
+  }
+}
+```
+
+If the repo already has scripts with those names, update them instead of duplicating.
+
+6. Update `.gitignore`.
+
+Add the entries from:
+
+- `node_modules/@sebastianandreasson/pi-autonomous-agents/templates/gitignore.fragment`
+
+Merge them into the repo `.gitignore` without duplicating existing lines.
+
+7. Pick a safe default verification command.
+
+Important:
+
+- `testCommand` must be fast and bounded.
+- Do not use a long end-to-end happy-path spec as the inner-loop default.
+- Prefer smoke tests or a narrow targeted command.
+
+If the repo does not yet have a good smoke command, set `testCommand` to an empty string and note that setup is incomplete.
+
+8. Configure models conservatively.
+
+Recommended pattern:
+
+- local model for `developer`
+- local model for `developerRetry`
+- local model for `developerFix`
+- local or slightly stronger model for `tester`
+- stronger frontier model for `visualReview` only if available
+
+Example shape:
+
+```json
+{
+  "piModel": "local/dev-model",
+  "visualReviewModel": "cloud/vision-model",
+  "roleModels": {
+    "developer": "local/dev-model",
+    "developerRetry": "local/dev-model",
+    "developerFix": "local/dev-model",
+    "tester": "local/tester-model",
+    "testerCommit": "local/tester-model",
+    "visualReview": "cloud/vision-model"
+  }
+}
+```
+
+9. Validate the setup.
+
+Run at least:
+
+```bash
+PI_CONFIG_FILE=pi.config.json pi-harness once
+```
+
+If the repo is not ready for a real run yet, at minimum run:
+
+```bash
+PI_CONFIG_FILE=pi.config.json PI_TRANSPORT=mock PI_TEST_CMD= pi-harness once
+```
+
+If setup validation fails, fix the config rather than leaving a half-configured repo.
+
+## Agent Rules
+
+- Reuse existing repo conventions where possible.
+- Do not replace project-specific instructions with generic text if good instructions already exist.
+- Do not invent fake test commands or model endpoints.
+- Do not enable visual review unless the repo actually has a usable capture command and model config.
+- Keep changes minimal and local to harness setup.
+
+## What To Report Back
+
+When setup is complete, report:
+
+- which files were created or updated
+- which verification command was configured
+- whether visual review was enabled
+- which roles were mapped to which models
+- whether validation was run successfully

package/docs/PI_SUPERVISOR.md

@@ -0,0 +1,246 @@
+# PI Harness Supervisor
+
+`pi-harness` provides a bounded unattended-work supervisor for TODO-driven local-agent loops.
+
+The package is generic. It orchestrates the loop, but each consuming repo defines its own:
+
+- `TODOS.md`
+- project instructions
+- verification command
+- visual capture flow
+- model/backend configuration
+
+## Core Flow
+
+Each real iteration follows this sequence:
+
+1. `developer` implements one coherent task from `TODOS.md`.
+2. A fast local verification command runs immediately after the developer round.
+3. If verification passes, `tester` reviews the change independently from a skeptical user-facing perspective.
+4. If tester or verification finds a real issue, the supervisor gives the findings back to `developer` for one focused repair pass.
+5. If tester reaches `PASS`, tester provides a commit plan and the harness performs the actual git finalization.
+6. Optionally, every `N` successful iterations, the harness runs a read-only visual review over screenshots and persists the feedback for later runs.
+7. If that visual review returns `FAIL`, `BLOCKED`, or times out, the iteration is not counted as a success and the feedback is carried into later prompts.
+
+## Package Contents
+
+Main package files:
+
+- `src/pi-supervisor.mjs`: controller
+- `src/pi-client.mjs`: transport layer
+- `src/pi-rpc-adapter.mjs`: built-in adapter from supervisor JSON to `pi --mode rpc`
+- `src/pi-config.mjs`: config loader
+- `src/pi-repo.mjs`: repo helpers, verification runner, git finalize step
+- `src/pi-telemetry.mjs`: telemetry writer/reader
+- `src/pi-prompts.mjs`: default prompt builders
+- `src/pi-visual-review.mjs`: multimodal visual-review worker
+- `src/pi-visual-once.mjs`: one-shot manual visual review runner
+- `src/pi-report.mjs`: telemetry summary report
+- `templates/DEVELOPER.md`: default developer-role instructions template
+- `templates/TESTER.md`: default tester-role instructions template
+
+## CLI
+
+```bash
+pi-harness once
+pi-harness run
+pi-harness report
+pi-harness visual-once
+```
+
+The package reads `PI_CONFIG_FILE` if provided. Otherwise it falls back to the bundled generic `pi.config.json`.
+
+## Config Contract
+
+Projects typically provide their own `pi.config.json` with fields such as:
+
+- `taskFile`
+- `developerInstructionsFile`
+- `testerInstructionsFile`
+- `roleModels`
+- `testCommand`
+- `continueAfterSeconds`
+- `toolContinueAfterSeconds`
+- `noEventTimeoutSeconds`
+- `toolNoEventTimeoutSeconds`
+- `visualCaptureCommand`
+- `visualFeedbackFile`
+- `testerFeedbackFile`
+- `models`
+- `piModel`
+- `visualReviewModel`
+
+Model entries may carry their own OpenAI-compatible endpoint settings, so the PI text loop and the multimodal visual reviewer can point at different backends without changing code.
+
+`piModel` is the default text model. Projects can optionally override specific roles through `roleModels`, for example:
+
+```json
+{
+  "piModel": "local/dev-model",
+  "visualReviewModel": "cloud/vision-model",
+  "roleModels": {
+    "developer": "local/dev-model",
+    "developerRetry": "local/dev-model",
+    "developerFix": "local/dev-model",
+    "tester": "local/tester-model",
+    "testerCommit": "local/tester-model",
+    "visualReview": "cloud/vision-model"
+  }
+}
+```
+
+This lets the main developer/tester loop stay on local models while a stronger frontier model is reserved for periodic review roles.
+
+For unattended inner-loop work, `testCommand` should be a bounded smoke gate rather than a long real-time end-to-end happy-path run. Reserve full-flow Playwright journeys for explicit nightly or post-run lanes.
+
+## Transport Contract
+
+The supervisor supports:
+
+- `PI_TRANSPORT=mock`
+- `PI_TRANSPORT=adapter`
+
+The built-in adapter command is typically:
+
+```bash
+pi-harness adapter
+```
+
+When using `adapter`, set `PI_ADAPTER_COMMAND` to a command that:
+
+1. Reads one JSON request from `stdin`
+2. Talks to PI RPC or your own PI wrapper
+3. Writes one JSON response to `stdout`
+4. Exits with code `0` on success
+
+Request shape:
+
+```json
+{
+  "sessionId": "existing-or-empty",
+  "sessionFile": "/absolute/path/to/session.jsonl",
+  "prompt": "controller prompt",
+  "cwd": "/absolute/repo/path",
+  "taskFile": "/absolute/repo/path/TODOS.md",
+  "instructionsFile": "/absolute/repo/path/pi/DEVELOPER.md",
+  "runtimeDir": "/absolute/repo/path/.pi-runtime",
+  "piCli": "pi",
+  "model": "local/model-name",
+  "tools": "read,bash,edit,write,grep,find,ls",
+  "thinking": "",
+  "noExtensions": false,
+  "noSkills": false,
+  "noPromptTemplates": false,
+  "noThemes": true,
+  "metadata": {
+    "iteration": 1,
+    "retryCount": 0,
+    "reason": "main_workflow"
+  }
+}
+```
+
+Response shape:
+
+```json
+{
+  "sessionId": "stable-session-id",
+  "sessionFile": "/absolute/path/to/session.jsonl",
+  "status": "success",
+  "output": "agent output text",
+  "notes": "short controller note"
+}
+```
+
+Allowed response `status` values:
+
+- `success`
+- `stalled`
+- `timed_out`
+- `failed`
+- `canceled`
+
+## Git Finalization
+
+The harness is designed to keep commit history structured:
+
+1. `developer` should leave a clean, reviewable diff and should not commit.
+2. `tester` should review functionality and, on `PASS`, provide a commit plan:
+   - `COMMIT_MESSAGE: ...`
+   - `COMMIT_FILES:`
+   - `- path/to/file`
+3. The harness stages only those requested files and performs the commit itself.
+4. If the requested plan cannot be isolated safely, the iteration is blocked or failed instead of committing unrelated work.
+
+## Persistent Handoffs
+
+The harness persists two cross-iteration handoff files:
+
+- visual review feedback:
+  - `pi-output/visual-review/FEEDBACK.md`
+- tester feedback:
+  - `pi-output/tester-feedback/FEEDBACK.md`
+
+These files are included in later developer/tester prompts, so new runs start with the latest review context.
+
+Commit-plan follow-up passes still write history entries, but they do not replace the latest substantive tester feedback file. That keeps later developer turns grounded in the last real functional review rather than commit bookkeeping.
+
+## Visual Capture Contract
+
+The visual-review layer is intentionally generic. The harness does not know how to navigate a specific project.
+
+Instead, when `PI_VISUAL_CAPTURE_CMD` is configured, it runs that command with:
+
+- `PI_VISUAL_ITERATION`
+- `PI_VISUAL_PHASE`
+- `PI_VISUAL_CAPTURE_DIR`
+- `PI_VISUAL_MANIFEST_FILE`
+- `PI_VISUAL_CHANGED_FILES`
+
+The capture command must write a JSON manifest at `PI_VISUAL_MANIFEST_FILE` with this shape:
+
+```json
+{
+  "screens": [
+    {
+      "id": "main_menu",
+      "label": "Main menu",
+      "path": "main-menu.png"
+    }
+  ]
+}
+```
+
+`path` is resolved relative to `PI_VISUAL_CAPTURE_DIR`. The harness validates that each referenced image exists before calling the multimodal visual reviewer.
+
+## Loop Mitigation
+
+The built-in adapter mitigates obvious local loops by watching PI RPC tool events:
+
+- repeated identical tool calls are aborted
+- repeated same-path churn is aborted
+- a soft `continue` can be sent after inactivity
+- a separate tool-aware watchdog can tolerate long-running `bash` or browser work without treating the turn as dead
+- a hard no-event timeout aborts a wedged turn instead of hanging indefinitely
+
+Important: terminal streaming does not reset the heartbeat by itself. The watchdog keys off PI RPC events and active tool state, not raw shell output.
+
+## Telemetry
+
+Each step records:
+
+- timestamp
+- iteration
+- phase
+- kind
+- status
+- transport
+- session id
+- timeout flag
+- exit code
+- duration
+- commit before and after
+- changed file count
+- verification status
+- retry count
+- notes

package/package.json

@@ -0,0 +1,37 @@
+{
+  "name": "@sebastianandreasson/pi-autonomous-agents",
+  "private": false,
+  "version": "0.1.0",
+  "type": "module",
+  "description": "Portable unattended PI harness for developer/tester/visual-review loops.",
+  "license": "MIT",
+  "exports": {
+    ".": "./src/index.mjs",
+    "./cli": "./src/cli.mjs",
+    "./pi-config": "./src/pi-config.mjs",
+    "./pi-supervisor": "./src/pi-supervisor.mjs",
+    "./pi-visual-review": "./src/pi-visual-review.mjs"
+  },
+  "bin": {
+    "pi-harness": "./src/cli.mjs"
+  },
+  "scripts": {
+    "check": "node --check src/cli.mjs && node --check src/pi-client.mjs && node --check src/pi-config.mjs && node --check src/pi-flow.mjs && node --check src/pi-heartbeat.mjs && node --check src/pi-prompts.mjs && node --check src/pi-repo.mjs && node --check src/pi-report.mjs && node --check src/pi-rpc-adapter.mjs && node --check src/pi-supervisor.mjs && node --check src/pi-telemetry.mjs && node --check src/pi-visual-once.mjs && node --check src/pi-visual-review.mjs && node --check src/index.mjs && node --check test/pi-heartbeat.test.mjs && node --check test/pi-role-models.test.mjs && node --check test/pi-flow.test.mjs",
+    "test": "node --test test/pi-heartbeat.test.mjs test/pi-role-models.test.mjs test/pi-flow.test.mjs"
+  },
+  "files": [
+    "src",
+    "templates",
+    "docs",
+    "pi.config.json",
+    "SETUP.md",
+    "README.md"
+  ],
+  "engines": {
+    "node": ">=20"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/sebastianandreasson/pi-autonomous-agents.git"
+  }
+}

package/pi.config.json

@@ -0,0 +1,28 @@
+{
+  "transport": "adapter",
+  "adapterCommand": "pi-harness adapter",
+  "instructionsFile": "",
+  "taskFile": "TODOS.md",
+  "streamTerminal": false,
+  "loopRepeatThreshold": 12,
+  "samePathRepeatThreshold": 8,
+  "continueAfterSeconds": 300,
+  "toolContinueAfterSeconds": 900,
+  "continueMessage": "continue",
+  "noEventTimeoutSeconds": 900,
+  "toolNoEventTimeoutSeconds": 1800,
+  "testCommand": "",
+  "visualFeedbackFile": "pi-output/visual-review/FEEDBACK.md",
+  "testerFeedbackFile": "pi-output/tester-feedback/FEEDBACK.md",
+  "testerFeedbackHistoryDir": "pi-output/tester-feedback/history",
+  "visualReviewHistoryDir": "pi-output/visual-review/history",
+  "visualCaptureDir": "pi-output/visual-capture",
+  "visualCaptureCommand": "",
+  "visualCaptureTimeoutSeconds": 300,
+  "visualReviewEnabled": false,
+  "visualReviewEveryNSuccesses": 5,
+  "visualReviewModel": "",
+  "visualReviewCommand": "pi-harness visual-review-worker",
+  "visualReviewMaxImages": 8,
+  "visualReviewTimeoutSeconds": 300
+}

package/src/cli.mjs

@@ -0,0 +1,48 @@
+#!/usr/bin/env node
+
+import path from 'node:path'
+import { spawn } from 'node:child_process'
+import process from 'node:process'
+import { fileURLToPath } from 'node:url'
+
+const scriptDir = path.dirname(fileURLToPath(import.meta.url))
+
+const COMMANDS = new Map([
+  ['once', 'pi-supervisor.mjs'],
+  ['run', 'pi-supervisor.mjs'],
+  ['report', 'pi-report.mjs'],
+  ['visual-once', 'pi-visual-once.mjs'],
+  ['adapter', 'pi-rpc-adapter.mjs'],
+  ['visual-review-worker', 'pi-visual-review.mjs'],
+])
+
+function main() {
+  const subcommand = process.argv[2] || 'once'
+  const scriptName = COMMANDS.get(subcommand)
+  if (!scriptName) {
+    console.error(`Unknown pi-harness command "${subcommand}". Expected one of: ${[...COMMANDS.keys()].join(', ')}`)
+    process.exitCode = 1
+    return
+  }
+
+  const childArgs = [path.join(scriptDir, scriptName)]
+  if (subcommand === 'once' || subcommand === 'run') {
+    childArgs.push(subcommand)
+  }
+
+  const child = spawn(process.execPath, childArgs, {
+    cwd: process.cwd(),
+    env: process.env,
+    stdio: 'inherit',
+  })
+
+  child.on('exit', (code, signal) => {
+    if (signal) {
+      process.kill(process.pid, signal)
+      return
+    }
+    process.exitCode = code ?? 1
+  })
+}
+
+main()

package/src/index.mjs

@@ -0,0 +1,7 @@
+export { loadConfig, resolveRoleModel, resolveRoleModelName } from './pi-config.mjs'
+export {
+  deriveFinalStatusWithVisualReview,
+  deriveWorkflowStatus,
+  shouldPersistLatestTesterFeedback,
+} from './pi-flow.mjs'
+export { runAgentTurn } from './pi-client.mjs'