neurastream 0.1.7 → 0.1.9

package/README.md

@@ -1,19 +1,54 @@
 # neurastream
 
-Agent orchestration (queue, guards, blockers, scoring, ledger) as one installable package. The core does not name a harness. Adapters translate harness events into core calls. The first adapter is omp.
+Guardrails for running many AI coding agents at once.
 
-## Install
+One "main" agent plans the work and reviews it; cheaper "helper" agents do the coding. neurastream is the set of checks between them. It keeps a queue of written plans, refuses helper actions a plan forbids, limits how many helpers run per provider, scores each helper by accepted work per dollar, and only lets a plan leave the queue once its branch is really merged.
+
+The core does not depend on one agent tool. Adapters translate a tool's events into core calls. The first adapter is [omp](https://github.com/can1357/oh-my-pi).
+
+```mermaid
+flowchart LR
+  O[Owner message] --> P[Plan file in docs/plans]
+  P --> Q[queue-check]
+  Q --> S[Spawn checks<br/>ledger cap, score gate, brief-lint]
+  S --> H[Helper agent<br/>blockers + call budget]
+  H --> R[Main review<br/>agent-score accept]
+  R --> M[Merge to main]
+  M --> C[queue-close<br/>merged proof + done check]
+```
+
+## What it does
 
-Do not install into the real home while this package is still being built. Use a sandbox home.
+| Part | What it enforces |
+|---|---|
+| Queue (`queue-check`, `queue-close`, `queue-cancel`) | Every open plan is listed in an index and follows a template. A plan leaves the queue only when its branch is merged into remote main and its done check passes; a failing check reopens it. A cancelled plan's unmerged work is saved under `refs/queue-archive/`. |
+| Blockers | Each plan's phase 0 lists what helpers may not do: push, touch a path, run a command, or write a text. The guard refuses those calls. |
+| Guards | Helpers may only edit their plan's `## Progress`; they get a warning at call 90 and are stopped at call 100. Protected files need an owner unlock (a file the owner touches, valid 30 minutes). |
+| Ledger (`agent-ledger`) | Tokens, cost, state and exit reason per agent; `--cap` refuses a spawn when a provider already runs 2. |
+| Scoring (`agent-score`) | Points only for work Main accepted by commit, times the phase difficulty. Ranked per dollar and per hour. |
+| Brief lint (`brief-lint`) | Refuses a helper brief that misses a required rule (acceptance, stop condition, no push, call budget). |
+| Machine health (`machine-guard`, `main-start`, `heavy-run`) | Load, free memory, disk and `/tmp` limits; one heavy test run at a time. |
+| Unlanded work (`wip-check`, `wip-snapshot`, `wip-backup`) | Lists branches on no remote and dirty worktrees; snapshots them as hidden commits; can push them to a backup repo. |
+| Session rollover | When the main session file passes 10 MB, the next owner message starts a fresh session from a handoff note. |
+
+## Requirements
+
+- [Bun](https://bun.sh) 1.x
+- git
+- omp, for the guards and rollover (the command-line tools work without it)
+
+## Install
 
 ```sh
-export HOME=/tmp/neurastream-home-dev
 bun add -g neurastream
 neurastream init
 neurastream adapter install omp
+neurastream doctor
 ```
 
-`init` writes `~/.config/neurastream/config.yml` from the defaults in `docs/config.md`, plus an empty index, notes, the queue template, blocker wrappers, and the state directories. A second run changes nothing. `adapter install omp` links the extension under that same home (`~/.omp/agent/extensions` and `omp plugin link`). Neither command touches `~/.omp` or `~/.local` of the real user.
+`init` writes `~/.config/neurastream/config.yml` from the defaults in [docs/config.md](docs/config.md), plus an empty index, notes, the queue template, blocker wrappers, and the state directories. A second run changes nothing. `adapter install omp` links the extension (`~/.omp/agent/extensions` and `omp plugin link`). `doctor` exits 0 when the install matches the package.
+
+The defaults are the author's setup (paths, the backup repo, provider names). Edit `config.yml` before relying on them. To try it without touching your real home, set `HOME=/tmp/neurastream-home-dev` first.
 
 Check the merged result:
 
@@ -21,11 +56,21 @@ Check the merged result:
 neurastream config --json
 ```
 
-Compatibility bins keep today's names (`agent-score`, `agent-ledger`, `brief-lint`, `oc-run`, `wip-check`, `wip-backup`, `wip-snapshot`, `pm-fail-facts`, `main-health`, `queue-check`, `queue-close`, `queue-cancel`, `main-start`, `machine-guard`, `glm-reset`, `priority-watch`, `resource-alerts`, `tmp-snapshot`, `heavy-run`). They are the same program, selected by the bin name.
+Every tool is one program, selected by the name it was called with: `agent-score`, `agent-ledger`, `brief-lint`, `oc-run`, `wip-check`, `wip-backup`, `wip-snapshot`, `pm-fail-facts`, `main-health`, `queue-check`, `queue-close`, `queue-cancel`, `main-start`, `machine-guard`, `glm-reset`, `priority-watch`, `resource-alerts`, `tmp-snapshot`, `heavy-run`.
+
+## Develop
+
+```sh
+bun install
+bun run typecheck
+bun test
+```
+
+Green is both exiting 0.
 
-Scoring: `agent-score accept` records accepted work by commit; `agent-score validate <Agent> --plan <plan.md> --phase <P> --evidence <report.md> --section "<heading prefix>"` records a validation phase (checks/tests ran, report section written, no commits) with proof Main cannot fake by typing: the plan line must be ticked `- [x]`, the cited section body must name a heavy-run label run by that agent (from `paths.heavy_log`), and the phase must not be scored already. The ranking board counts it in `Accepted (diag/val)` as `N (d/v)`.
+## Scoring details
 
-Green is `bun run typecheck` and `bun test`, both exit 0.
+`agent-score accept` records accepted work by commit; `agent-score validate <Agent> --plan <plan.md> --phase <P> --evidence <report.md> --section "<heading prefix>"` records a validation phase (checks/tests ran, report section written, no commits) with proof Main cannot fake by typing: the plan line must be ticked `- [x]`, the cited section body must name a heavy-run label run by that agent (from `paths.heavy_log`), and the phase must not be scored already. The ranking board counts it in `Accepted (diag/val)` as `N (d/v)`.
 
 ## Main reads the code for its plans
 
@@ -51,6 +96,7 @@ The switch is never automatic:
 - The owner's next plain message then closes the session and starts a fresh one. The fresh session opens with the note verbatim, the generated `main-start` state (including the owner holds), and then the owner's message. The previous session file stays where it is.
 - While own jobs (subagents included) are still running, the switch waits: a rollover would cancel them, so the owner's message stays in the current session until they finish.
 - `/main-rollover <message>` does the same by hand. It refuses below the threshold, without a fresh note, or while jobs run.
+- `/main-rollover` shows a notice at once and gathers `main-start` in the background, so omp keeps repainting and reading keys until the new session opens.
 - `rollover.enabled: false` turns the feature off.
 
 ## Changing a script
@@ -63,3 +109,7 @@ neurastream adapter install omp
 ```
 
 Live copies are never hand-edited. `neurastream doctor` names each hand-edited or foreign installed file, and a stale install (the installed version is not `npm view neurastream version`). If the registry cannot be reached, that check is a warning, not a failure.
+
+## License
+
+MIT. See [LICENSE](LICENSE).

package/package.json

@@ -1,13 +1,19 @@
 {
   "name": "neurastream",
-  "version": "0.1.7",
-  "description": "Harness-agnostic agent orchestration: queue, guards, blockers, scoring, ledger.",
+  "version": "0.1.9",
+  "description": "Guardrails for running many AI coding agents at once: plan queue, blockers, call budgets, provider caps, scoring, merge-verified close.",
+  "keywords": ["ai-agents", "coding-agents", "orchestration", "guardrails", "omp", "bun", "cli"],
+  "homepage": "https://github.com/umiddey/neurastream#readme",
+  "bugs": "https://github.com/umiddey/neurastream/issues",
   "type": "module",
   "license": "MIT",
   "repository": {
     "type": "git",
     "url": "git+https://github.com/umiddey/neurastream.git"
   },
+  "engines": {
+    "bun": ">=1.0"
+  },
   "publishConfig": {
     "access": "public"
   },

package/src/adapters/omp/index.ts

@@ -432,7 +432,8 @@ export function createOmpAdapter(pi: ExtensionHost, cfg: Config): void {
 				c.notify(`main-rollover: own jobs are still running (${c.runningJobs().join(", ")}); a rollover would cancel them.`);
 				return;
 			}
-			const generated = [await mainStartText([], cfg), holdsBlock(q.ownerHolds())].join("\n");
+			c.notify("main-rollover: collecting main-start state for the handoff (this can take a minute)…");
+			const generated = [await mainStartText([], cfg, { inheritStdin: false }), holdsBlock(q.ownerHolds())].join("\n");
 			const handoff = buildHandoff({
 				generated,
 				note: readFileSync(path, "utf8"),

package/src/adapters/omp/sessions.ts

@@ -54,17 +54,10 @@ function costOf(m: Record<string, unknown>): number {
 	return c && typeof c.total === "number" ? c.total : 0;
 }
 
-interface Built {
-	run: Run;
-	mtime: number;
-}
-
-function build(file: string, cfg: Config): Built | null {
+function build(file: string, cfg: Config, mtime: number): Run | null {
 	let text: string;
-	let mtime: number;
 	try {
 		text = readFileSync(file, "utf8");
-		mtime = statSync(file).mtimeMs / 1000;
 	} catch {
 		return null;
 	}
@@ -202,7 +195,7 @@ function build(file: string, cfg: Config): Built | null {
 		scoreSurvived: done && !scoreErr,
 		scoreEnd: scoreErr || (done ? "done" : "no final message"),
 	};
-	return { run, mtime };
+	return run;
 }
 
 function readMain(file: string): MainSession {
@@ -253,9 +246,15 @@ export function ompRunSource(root: string, cfg: Config): RunSource {
 			const files = [...jsonlAt(root, 1), ...jsonlAt(root, 2)];
 			const runs: Run[] = [];
 			for (const file of files) {
-				const built = build(file, cfg);
-				if (!built || built.mtime < since) continue;
-				runs.push(built.run);
+				let mtime: number;
+				try {
+					mtime = statSync(file).mtimeMs / 1000;
+				} catch {
+					continue;
+				}
+				if (mtime < since) continue;
+				const run = build(file, cfg, mtime);
+				if (run) runs.push(run);
 			}
 			return runs;
 		},

package/src/core/agent-ledger.ts

@@ -64,13 +64,21 @@ export function agentLedger(args: string[], cfg: Config, source: RunSource): num
 
 	if (!args.includes("--cap")) return 0;
 	const live: Record<string, string[]> = {};
+	const notLive: string[] = [];
 	for (const r of rows) {
 		if ((r.statusDetail.startsWith("RUNNING") || r.statusDetail.startsWith("WAITING")) && !r.id.startsWith("MAIN")) {
+			if ((nowSec() - r.lastWriteAt) / 60 >= cfg.ledger.idle_not_live_min) {
+				notLive.push(r.id);
+				continue;
+			}
 			const provider = r.model.split("/")[0] ?? "";
 			(live[provider] ??= []).push(r.id);
 		}
 	}
 	process.stdout.write(`running per provider: ${pyDictInt(Object.fromEntries(Object.entries(live).map(([k, v]) => [k, v.length])))}\n`);
+	if (notLive.length) {
+		process.stdout.write(`NOT LIVE (no write for ≥ ${cfg.ledger.idle_not_live_min}m, not counted): ${notLive.join(", ")}\n`);
+	}
 	let full = Object.entries(live).filter(([, v]) => v.length >= cfg.ledger.max_running_per_provider).map(([k]) => k);
 	for (const k of full) process.stdout.write(`FULL ${k}: ${live[k]!.join(", ")}\n`);
 	const over = rows.filter(r => r.statusDetail.includes("OVER-BUDGET")).map(r => r.id);

package/src/core/main-start.ts

@@ -1,5 +1,5 @@
 import { readFileSync } from "node:fs";
-import { spawnSync } from "node:child_process";
+import { spawn } from "node:child_process";
 import { dirname, join } from "node:path";
 import { fileURLToPath } from "node:url";
 import type { Config } from "./config.ts";
@@ -8,39 +8,83 @@ function bin(name: string, cfg: Config): string {
 	return join(cfg.paths.home, ".local/bin", name);
 }
 
+interface RunOpts {
+	stdin?: "inherit" | "ignore" | "pipe";
+	/** Written to the child's stdin when stdin is "pipe". */
+	input?: string;
+	stderr?: "inherit" | "pipe" | "ignore";
+}
+
+interface RunResult {
+	stdout: string;
+	stderr: string;
+	/** Set when the child could not start (e.g. ENOENT). */
+	error?: Error;
+}
+
+// Runs a child without blocking the caller's event loop. Resolves once the child has closed its
+// streams, or immediately when it fails to start. The handoff runs inside a live agent session,
+// which must keep handling events while Main's state is gathered.
+function run(file: string, args: string[], opts: RunOpts = {}): Promise<RunResult> {
+	const { promise, resolve } = Promise.withResolvers<RunResult>();
+	const child = spawn(file, args, {
+		stdio: [opts.stdin ?? "ignore", "pipe", opts.stderr ?? "pipe"],
+	});
+	let stdout = "";
+	let stderr = "";
+	child.stdout?.setEncoding("utf8").on("data", chunk => {
+		stdout += chunk;
+	});
+	child.stderr?.setEncoding("utf8").on("data", chunk => {
+		stderr += chunk;
+	});
+	// A filter such as awk may exit before it has read all its input.
+	child.stdin?.on("error", () => {});
+	if (opts.stdin === "pipe") child.stdin?.end(opts.input ?? "");
+	child.on("error", error => resolve({ stdout, stderr, error }));
+	child.on("close", () => resolve({ stdout, stderr }));
+	return promise;
+}
+
 // Main's session-start preflight as one text. The CLI prints it; the session rollover handoff
 // (src/core/rollover.ts) carries the same text into the new session. Children's stdout is
 // collected in order, so the printed output matches the original script line for line.
-export function mainStartText(args: string[], cfg: Config): string {
+// inheritStdin: the CLI passes its terminal to the children that ask the operator something;
+// an agent host passes false, because its stdin is the session's input.
+// stderr: the CLI lets a child's stderr reach the terminal. An agent host captures it and appends it
+// after that child's stdout, because a write to the terminal would draw over the session's UI.
+export async function mainStartText(args: string[], cfg: Config, opts: { inheritStdin: boolean }): Promise<string> {
 	const out: string[] = [];
 	const say = (line: string) => out.push(`${line}\n`);
 	const h = args[0] ?? String(cfg.health.default_hours);
+	const childStdin = opts.inheritStdin ? "inherit" : "ignore";
+	const childStderr = opts.inheritStdin ? "inherit" : "pipe";
 	say("=== machine");
-	const mg = spawnSync(bin("machine-guard", cfg), [], { encoding: "utf8", env: process.env });
-	const mgLines = (mg.stdout ?? "").trim().split("\n");
+	const mg = await run(bin("machine-guard", cfg), [], { stderr: "ignore" });
+	const mgLines = mg.stdout.trim().split("\n");
 	say(mgLines.at(-1) ?? "");
 	say(readFileSync("/proc/loadavg", "utf8").trim());
 	say("");
 	say(`=== agents (last ${h}h; DIED = check its worktree, relaunch FRESH with its progress file)`);
-	const ledger = spawnSync(bin("agent-ledger", cfg), ["--hours", h], { encoding: "utf8", env: process.env });
-	const awk = spawnSync(
+	const ledger = await run(bin("agent-ledger", cfg), ["--hours", h], { stderr: "ignore" });
+	const awk = await run(
 		"awk",
 		["NR==1 || /DIED/ {print; next} {ok[++n]=$0} END {for (i=1;i<=n;i++) print ok[i]}"],
-		{ input: ledger.stdout ?? "", encoding: "utf8" },
+		{ stdin: "pipe", input: ledger.stdout, stderr: "ignore" },
 	);
-	out.push(awk.stdout ?? "");
-	const cap = spawnSync(bin("agent-ledger", cfg), ["--cap"], { encoding: "utf8", env: process.env, stdio: ["ignore", "pipe", "ignore"] });
-	for (const line of (cap.stdout ?? "").split("\n")) {
+	out.push(awk.stdout);
+	const cap = await run(bin("agent-ledger", cfg), ["--cap"], { stderr: "ignore" });
+	for (const line of cap.stdout.split("\n")) {
 		if (/^(PRIORITY IDLE|FULL)/.test(line)) say(line);
 	}
 	say("");
 	say("=== main thread");
-	const health = spawnSync(bin("main-health", cfg), [], { encoding: "utf8", env: process.env, stdio: ["inherit", "pipe", "inherit"] });
-	out.push(health.stdout ?? "");
+	const health = await run(bin("main-health", cfg), [], { stdin: childStdin, stderr: childStderr });
+	out.push(health.stdout + health.stderr);
 	say("");
 	say("=== queue");
-	const check = spawnSync(bin("queue-check", cfg), [], { encoding: "utf8", env: process.env, stdio: ["inherit", "pipe", "inherit"] });
-	out.push(check.stdout ?? "");
+	const check = await run(bin("queue-check", cfg), [], { stdin: childStdin, stderr: childStderr });
+	out.push(check.stdout + check.stderr);
 	say("");
 	say("=== heavy-run lock");
 	try {
@@ -52,14 +96,14 @@ export function mainStartText(args: string[], cfg: Config): string {
 	// The CLI file itself (shebang `#!/usr/bin/env bun`), never process.execPath: inside a host
 	// harness that is the harness binary, which would start a new agent session instead.
 	const cli = join(dirname(fileURLToPath(import.meta.url)), "../cli/main.ts");
-	const doctor = spawnSync(cli, ["doctor"], { encoding: "utf8", env: process.env });
-	const text = `${doctor.stdout ?? ""}${doctor.stderr ?? ""}`;
+	const doctor = await run(cli, ["doctor"]);
+	const text = `${doctor.stdout}${doctor.stderr}`;
 	out.push(text.endsWith("\n") || text === "" ? text : `${text}\n`);
 	if (doctor.error) say(doctor.error.message);
 	return out.join("");
 }
 
-export function mainStart(args: string[], cfg: Config): number {
-	process.stdout.write(mainStartText(args, cfg));
+export async function mainStart(args: string[], cfg: Config): Promise<number> {
+	process.stdout.write(await mainStartText(args, cfg, { inheritStdin: true }));
 	return 0;
 }