headlesscode 1.0.2 → 1.2.0

package/README.md

@@ -1,66 +1,63 @@
 # headlesscode
 
-A small, purpose-built, genuinely headless coding-agent harness. It reuses what
-makes [Zoo Code](https://github.com/Zoo-Code-Org/Zoo-Code) (an Apache-2.0 Roo
-Code fork) good — its system prompts, native tool schemas, and
-mode/rules configuration format — without carrying along the VS Code GUI
-dependency.
-
-This repo implements:
-
-- The **headless runtime engine** — vendoring the portable Apache-2.0 core and
-  the headless runtime (OpenRouter client, tool executor, tool-call parser,
-  orchestration loop, CLI).
-- A **headless orchestration layer** — split an issue into worker groups, run
-  each as a harness subprocess in its own git worktree, review, and merge
-  (see [`docs/phase2-orchestration.md`](./docs/phase2-orchestration.md)).
-- A **memory subsystem** — per-project knowledge facts + rolling session
-  summaries, with local semantic recall and a pluggable `MemoryStore`
-  contract for a remote backend.
-- **Headless QA** — run the target repo's `qa-agent` mode as a second harness
-  session (see [`docs/phase4-qa.md`](./docs/phase4-qa.md)) and a
-  human-approval deploy gate in front of the repo's `deploy-production.sh`
-  (see [`docs/phase4-deploy-gate.md`](./docs/phase4-deploy-gate.md)).
-- A **GitHub issue watcher** (see
-  [`docs/phase5-issue-watcher.md`](./docs/phase5-issue-watcher.md)) — file an
-  issue with a target label and a poll loop automatically splits and spawns it
-  through the existing orchestration pipeline, with durable idempotency so a
-  restart never double-spawns.
-- **Cloud-scaling guardrails** — a hard concurrent-session cap, a per-session
-  cost/time/iteration budget, and a `CloudProvider` abstraction (container/VM
-  per issue behind the same lifecycle as the local worktree flow) with an
-  evaluation-only cloud-provider sketch (see
-  [`docs/phase6-cloud.md`](./docs/phase6-cloud.md)). No live cloud resources
-  are launched; the caps/budgets land before any scaling.
-
-> ⚠️ **Security warning: default-allow arbitrary command execution.**
-> `headlesscode` is a headless coding agent: by default it runs **arbitrary
-> shell commands with your full user privileges** (no sandbox, no approval
-> prompts) and can read and modify your files — including `~/.ssh`, `~/.aws`,
-> and any other credentials your user can access. Run it only on machines and
-> with tasks you trust, and isolate it (container/VM/dedicated user) whenever
-> untrusted content is involved. See [`SECURITY.md`](./SECURITY.md) for the
-> full disclosure and the optional permissions layer that provides defense in
-> depth (not a security boundary).
-
-## Purpose
-
-Drive a coding agent headlessly against real git repos: read a target repo's
-`.roomodes` and `.roo/rules-<slug>/` files, build a system prompt from mode +
-rules, call an LLM API (OpenRouter in Phase 1) with the tool schema, execute
-tool calls via plain `fs`/`child_process` (no `vscode.*` anywhere), and loop
-until completion. Non-interactive by design.
+[![CI](https://github.com/Capsize-Games/headlesscode/actions/workflows/ci.yml/badge.svg)](https://github.com/Capsize-Games/headlesscode/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/headlesscode?logo=npm)](https://www.npmjs.com/package/headlesscode)
+[![Node.js >=18](https://img.shields.io/badge/node-%3E%3D18-339933?logo=node.js&logoColor=white)](https://nodejs.org/)
+[![License: Apache-2.0](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](./LICENSE)
+
+Run a coding agent where the work actually happens: in a terminal, a CI job,
+or an isolated git worktree. `headlesscode` brings the useful parts of
+[Zoo Code](https://github.com/Zoo-Code-Org/Zoo-Code) to a plain Node.js process,
+without requiring a VS Code window.
+
+It is made for real repositories and real engineering loops. The harness reads
+the target project's modes and rules, gives the model a controlled tool set,
+keeps work separated in git worktrees, and leaves behind logs and state that a
+person can inspect.
+
+## Why use it?
+
+- **Run repeatable repository tasks.** Give it a task or GitHub issue and let
+  it work through files, commands, and tests from a non-interactive process.
+- **Keep parallel work organized.** Split issues into worker groups, run them
+  in separate worktrees, review the results, and optionally run QA before a
+  human-approved deploy.
+- **Remember the project.** Opt-in local memory stores project facts and
+  rolling session summaries, with a pluggable storage boundary for a future
+  remote backend.
+- **Bound the expensive parts.** Per-session cost, duration, iteration, and
+  fleet-concurrency limits are built into the orchestration and watcher paths.
+- **Experiment with improvement.** The `improve` command runs a bounded
+  recursive self-improvement loop against an external evaluator, with the
+  default worker using a local Qwen 3.5 9B model through Ollama.
+
+The project also includes a GitHub issue watcher and an evaluation-only cloud
+provider interface. No live cloud resources are launched by the current
+implementation.
+
+> **Security warning: default-allow arbitrary command execution.**
+> By default, `headlesscode` runs arbitrary shell commands with the invoking
+> user's privileges. It can read and modify files, including credentials such
+> as `~/.ssh` and `~/.aws`, without approval prompts. Use it only with trusted
+> tasks and isolate it with a container, VM, or dedicated user when untrusted
+> content is involved. The optional permissions layer is defense in depth, not
+> a security boundary. See [`SECURITY.md`](./SECURITY.md).
 
 ## Quick start
 
-`headlesscode` is not yet published to the npm registry, so install it from a
-checkout:
+Install from npm:
 
 ```bash
-git clone https://github.com/Capsize-Games/headlesscode.git
-cd headlesscode
-npm install
+npm install -g headlesscode
+```
+
+Or run it without installing, via npx:
+
+```bash
+npx headlesscode --task "Fix the bug in src/index.ts" --workspace /path/to/target/repo
+```
 
+```bash
 # Required (except for --dry-run):
 export HEADLESSCODE_OPENROUTER_API_KEY=sk-or-...
 
@@ -70,7 +67,15 @@ export OPENROUTER_HTTP_REFERER=https://example.com       # OpenRouter app header
 export OPENROUTER_APP_TITLE="headlesscode"               # OpenRouter X-Title header
 export HEADLESSCODE_WORKSPACE_ROOT=/path/to/target/repo  # default workspace root
 
-# Run a task against a target repo, straight from the checkout:
+headlesscode --task "Fix the bug in src/index.ts" --workspace /path/to/target/repo
+```
+
+To work from a source checkout instead (for contributing):
+
+```bash
+git clone https://github.com/Capsize-Games/headlesscode.git
+cd headlesscode
+npm install
 node bin/headlesscode.mjs --task "Fix the bug in src/index.ts" --workspace /path/to/target/repo
 ```
 
@@ -134,6 +139,45 @@ printed to stdout. Exit code 0 = success; 1 = task failed (max iterations or
 consecutive-mistake limit); 2 = usage/config error (e.g. missing
 `HEADLESSCODE_OPENROUTER_API_KEY`).
 
+## Recursive self-improvement
+
+`headlesscode improve` is a bounded research loop for improving the harness
+itself. A supervisor creates isolated candidate worktrees, asks the local Qwen
+3.5 9B worker to make focused changes, runs regression plus visible and hidden
+evaluations, and keeps the archive, score, and selection decision outside the
+candidate worktree. Each generation also produces a report for human review.
+
+```bash
+# Inspect the planned experiment without creating worktrees or calling a model:
+npx tsx src/cli.ts improve --repo . --dry-run
+
+# Run one small generation with the local Ollama model:
+npx tsx src/cli.ts improve --repo . --population 2 --generations 1
+```
+
+This is an experiment, not unattended model training and not a security
+boundary. The current loop records model-candidate and training interfaces,
+trajectory datasets, parent-selection policies, and resource-tagged jobs, but
+does not yet train adapters, schedule multiple trajectories, or provide
+OS-level candidate isolation. Those limits are tracked in [the RSI roadmap](#rsi-roadmap).
+
+## RSI roadmap
+
+The implemented loop is intentionally honest about what remains. Follow-up
+work is tracked in GitHub:
+
+- [OS-level candidate sandbox](https://github.com/Capsize-Games/headlesscode/issues/3)
+- [Cryptographically verifiable evaluator and artifacts](https://github.com/Capsize-Games/headlesscode/issues/4)
+- [Resource-aware resumable scheduler](https://github.com/Capsize-Games/headlesscode/issues/5)
+- [Adaptive multi-trajectory search](https://github.com/Capsize-Games/headlesscode/issues/6)
+- [Validated curriculum fixtures](https://github.com/Capsize-Games/headlesscode/issues/7)
+- [Adversarial evaluation and cross-model supervision](https://github.com/Capsize-Games/headlesscode/issues/8)
+- [Real LoRA or QLoRA backend](https://github.com/Capsize-Games/headlesscode/issues/9)
+
+See [`docs/recursive-self-improvement.md`](./docs/recursive-self-improvement.md)
+for the design and [`docs/rsi-progress.md`](./docs/rsi-progress.md) for the
+record of the first bounded runs.
+
 ## Registering a new project
 
 To use headlesscode against a project that isn't set up as a headlesscode

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "headlesscode",
-	"version": "1.0.2",
+	"version": "1.2.0",
 	"description": "Standalone headless coding-agent harness: runs the Zoo Code agent loop (prompts, tools, modes) without a VS Code UI, driven by CLI, HTTP, and parallel worktree orchestration.",
 	"license": "Apache-2.0",
 	"repository": {

package/src/cli.ts

@@ -43,6 +43,7 @@ import { analyzeCliMain } from "./orchestrator/analyze-cli.js"
 import { costHistoryCliMain } from "./orchestrator/cost-history-cli.js"
 import { resolvePermissions, type PermissionsConfig } from "./permissions/config.js"
 import { resolveModelForMode, resolveReasoningEffortForMode } from "./config/mode-models.js"
+import { improveMain } from "./rsi/controller.js"
 
 const VERSION = "0.1.0"
 
@@ -166,6 +167,16 @@ interface CliOptions {
 	autoApproveModeSwitch: boolean
 	/** switch_mode: hard cap on total in-place mode switches per session (default 5). */
 	maxModeSwitches?: number
+	/**
+	 * Evidence-gated completion (fabrication fix, 2026-09-01): when set,
+	 * attempt_completion is refused unless every machine-checkable claim in
+	 * its result is independently verified against ground truth (file
+	 * existence, real command re-runs, serial logs, git history — see
+	 * src/engine/claims.ts). Default ON for the local code backend; this
+	 * flag forces it on for cloud sessions too. Also settable via
+	 * HEADLESSCODE_REQUIRE_EVIDENCE.
+	 */
+	requireEvidence: boolean
 }
 
 const USAGE = `headlesscode — headless coding-agent harness (Phase 1 engine)
@@ -176,6 +187,10 @@ Usage:
   headlesscode --dry-run [options]          # build system prompt + validate config, no LLM call
 
 Subcommands:
+  headlesscode improve --repo <path> [--model <id>] [--population <n>] [--dry-run]
+              Run a bounded recursive self-improvement generation. Candidates
+              use isolated git worktrees; evaluator/scoring paths are protected.
+              The default worker is the local Qwen 3.5 9B model.
   headlesscode orchestrate --repo <path> --issue <n>... [--qa] [--deploy] [--dry-run]
               Run a full parallel orchestration round
               (split → spawn → review → QA → deploy gate). See
@@ -389,6 +404,15 @@ Options:
                                granting itself a broader mode's edit
                                permissions. Also settable via
                                $HEADLESSCODE_AUTO_APPROVE_MODE_SWITCH
+  --require-evidence          Evidence-gated completion (fabrication fix):
+                               refuse attempt_completion unless every
+                               machine-checkable claim in its result is
+                               independently verified against ground truth
+                               (file existence, real command re-runs, serial
+                               logs, git history). Default ON for the local
+                               code backend; this forces it on for cloud
+                               sessions too. Also settable via
+                               $HEADLESSCODE_REQUIRE_EVIDENCE
   --max-mode-switches <n>    switch_mode: hard cap on total in-place mode
                                switches per session (default: 5 — see
                                DEFAULT_MAX_MODE_SWITCHES in src/engine/loop.ts).
@@ -415,6 +439,7 @@ Environment:
   HEADLESSCODE_STREAM          Opt-in SSE streaming ("1"/"true"/"yes"/"on")
   HEADLESSCODE_LOCAL_EXPLORE   Opt-in local exploration phase ("1"/"true")
   HEADLESSCODE_AUTO_APPROVE_MODE_SWITCH  Auto-approve switch_mode calls ("1"/"true"/"yes"/"on")
+  HEADLESSCODE_REQUIRE_EVIDENCE  Force evidence-gated completion ("1"/"true"/"yes"/"on")
   HEADLESSCODE_MAX_MODE_SWITCHES      switch_mode: hard cap on total in-place
                                mode switches per session (positive int)
   HEADLESSCODE_LOCAL_EXPLORE_MODEL    Local model (default qwen3.5:9b)
@@ -441,6 +466,7 @@ export function parseArgs(argv: string[]): { options: CliOptions; error?: string
 		stream: false,
 		localExplore: false,
 		autoApproveModeSwitch: false,
+		requireEvidence: false,
 	}
 
 	for (let i = 0; i < argv.length; i++) {
@@ -659,6 +685,9 @@ export function parseArgs(argv: string[]): { options: CliOptions; error?: string
 			case "--auto-approve-mode-switch":
 				options.autoApproveModeSwitch = true
 				break
+			case "--require-evidence":
+				options.requireEvidence = true
+				break
 			case "--checkpoint-dir": {
 				const value = next()
 				if (value === undefined) {
@@ -692,6 +721,13 @@ export function parseArgs(argv: string[]): { options: CliOptions; error?: string
 }
 
 export async function main(argv: string[] = process.argv.slice(2)): Promise<number> {
+	// Bounded recursive self-improvement: the supervisor owns the evaluator,
+	// archive, and selection logic while each candidate runs in its own
+	// worktree. See docs/recursive-self-improvement.md.
+	if (argv[0] === "improve") {
+		return improveMain(argv.slice(1))
+	}
+
 	// Phase 2 subcommand: `headlesscode orchestrate ...` — delegates to the
 	// orchestrator module (split → spawn → watch → review). Keeps the Phase 1
 	// run path untouched.
@@ -1155,6 +1191,26 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise<numb
 	// loop.ts's HeadlessSessionConfig.verifyBeforeCompletion doc comment.
 	const verifyBeforeCompletion =
 		useLocalCodeBackend && !envBoolean("HEADLESSCODE_ALLOW_UNVERIFIED_COMPLETION")
+	// Evidence-gated completion (fabrication fix, 2026-09-01): when on,
+	// attempt_completion is refused unless every machine-checkable claim in
+	// its result (a file exists, a specific command passed, serial markers
+	// appear, a PR exists) is independently verified against ground truth —
+	// the real filesystem, a real re-run of the exact command, the newest
+	// serial log, and real git history (see src/engine/claims.ts). This is
+	// the structural backstop for the FINAL_REPORT's central finding (§4): a
+	// session claimed "all three hard gates pass" with a fabricated serial
+	// excerpt when the driver was never merged and the claimed target didn't
+	// exist. Default ON for the local code backend (the finetune harness and
+	// real acceptance gates run local) unless explicitly disabled via
+	// HEADLESSCODE_ALLOW_UNVERIFIED_COMPLETION (same opt-in-override pattern
+	// as verifyBeforeCompletion); explicitly forceable via --require-evidence
+	// OR HEADLESSCODE_REQUIRE_EVIDENCE for cloud sessions too (the pure
+	// resolver also honors the env var — see resolveEvidenceRequiredCompletion).
+	const evidenceRequiredCompletion = resolveEvidenceRequiredCompletion(
+		options.requireEvidence,
+		process.env,
+		useLocalCodeBackend,
+	)
 	// A local (Qwen3.5-9B) session was observed live 2026-08-27 doing the
 	// actual work correctly (a real, correct edit_file call) and then dying
 	// anyway: its first attempt_completion was deferred (a prior
@@ -1208,6 +1264,19 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise<numb
 	// doc comment and executor.ts's largeOverwriteRefusal.
 	const guardLargeOverwrites =
 		useLocalCodeBackend && !envBoolean("HEADLESSCODE_ALLOW_UNGUARDED_OVERWRITES")
+	// 2026-09-02: real, confirmed, live-observed failure -- the read_file/
+	// list_files session cache's "[cache] unchanged, reuse the earlier
+	// result" short-circuit (src/tools/executor.ts) saves real token cost
+	// against a remote model's per-token bill, but against the local
+	// backend it directly produced a fabricated attempt_completion: an
+	// edit_file failure told a session to re-read and retry, it DID call
+	// read_file again exactly as instructed, got the cache-hit notice
+	// instead of real content, never made the SECOND identical call that
+	// would have returned real content again (the mechanism's own safety
+	// valve), and gave up with a false success claim instead. See loop.ts's
+	// HeadlessSessionConfig.disableReadFileCache doc comment.
+	const disableReadFileCache =
+		useLocalCodeBackend && !envBoolean("HEADLESSCODE_ALLOW_READ_FILE_CACHE_HIT")
 	// The cloud-tuned condensation defaults (0.75 hard threshold, 0.6 early-fire
 	// — see condense.ts) were measured firing needlessly aggressively against a
 	// local model: a trial condensing at 38 messages against a 16384-token
@@ -1359,7 +1428,16 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise<numb
 	// pattern already used for condenseThresholdFraction below: a runaway
 	// generation's blast radius should be a small fraction of the REAL
 	// local context window, not up to half of it.
-	const LOCAL_MAX_TOKENS = 8192
+	// 2026-09-02: 8192 was still too generous in practice -- verified live,
+	// repeatedly (joeos_finetune_data's eval_verifier.py ground-truth runs,
+	// same day): a stuck local-model turn reliably ran to the FULL 8192-token
+	// cap every time, taking 8-9 real minutes at this model's ~15 tok/s and
+	// consuming the entire session's remaining time budget without ever
+	// producing a tool call. Real, productive turns in the same logs (a tool
+	// call + a few sentences of reasoning) topped out around 1000-2000
+	// output tokens. Lowered so a stuck turn gets cut off in well under a
+	// minute instead of silently eating the whole budget.
+	const LOCAL_MAX_TOKENS = 2048
 	const maxTokens =
 		options.maxTokens ?? envNumber("HEADLESSCODE_MAX_TOKENS") ?? (useLocalCodeBackend ? LOCAL_MAX_TOKENS : undefined)
 
@@ -1406,12 +1484,14 @@ export async function main(argv: string[] = process.argv.slice(2)): Promise<numb
 		requireExplicitCompletion,
 		patchLocalToolSchemas,
 		verifyBeforeCompletion,
+		evidenceRequiredCompletion,
 		trackCost,
 		requireArtifactBeforeCompletion,
 		requireArtifactPathPattern: options.requireArtifactPath,
 		requireArtifactMinCitations: options.requireArtifactMinCitations,
 		requireArtifactSections: options.requireArtifactSections,
 		guardLargeOverwrites,
+		disableReadFileCache,
 		llmClient: client,
 		logger,
 		memory,
@@ -1496,15 +1576,46 @@ function envNumber(name: string): number | undefined {
 	return Number.isFinite(n) && n > 0 ? n : undefined
 }
 
-/** Parse an env boolean opt-in: "1"/"true"/"yes"/"on" → true; anything else (incl. unset) → false. */
-export function envBoolean(name: string): boolean {
-	const raw = process.env[name]
+/** Pure env-boolean parse: "1"/"true"/"yes"/"on" → true; anything else (incl. unset) → false. */
+export function envBooleanValue(name: string, env: NodeJS.ProcessEnv): boolean {
+	const raw = env[name]
 	if (raw === undefined || raw === "") {
 		return false
 	}
 	return ["1", "true", "yes", "on"].includes(raw.trim().toLowerCase())
 }
 
+/** Parse an env boolean opt-in from process.env: "1"/"true"/"yes"/"on" → true; anything else (incl. unset) → false. */
+export function envBoolean(name: string): boolean {
+	return envBooleanValue(name, process.env)
+}
+
+/**
+ * Resolve whether evidence-gated completion is ON for a session. Pure
+ * (flag + env + local-backend in, boolean out) so the full wiring is
+ * testable without spinning up a session.
+ *
+ * Order of precedence:
+ *   1. the explicit --require-evidence flag always wins;
+ *   2. HEADLESSCODE_REQUIRE_EVIDENCE env forces it on (cloud sessions
+ *      included — this is the finetune-harness/acceptance-gate hook, plan
+ *      A5: "force evidence-gated completion without code changes");
+ *   3. the local code backend defaults it ON unless explicitly disabled
+ *      via HEADLESSCODE_ALLOW_UNVERIFIED_COMPLETION (same opt-in-override
+ *      pattern as verifyBeforeCompletion).
+ */
+export function resolveEvidenceRequiredCompletion(
+	requireEvidenceFlag: boolean,
+	env: NodeJS.ProcessEnv,
+	useLocalCodeBackend: boolean,
+): boolean {
+	return (
+		requireEvidenceFlag ||
+		envBooleanValue("HEADLESSCODE_REQUIRE_EVIDENCE", env) ||
+		(useLocalCodeBackend && !envBooleanValue("HEADLESSCODE_ALLOW_UNVERIFIED_COMPLETION", env))
+	)
+}
+
 /**
 	* Resolve the Phase 3 memory store. Default OFF (null) to preserve existing
 	* behavior; enabled by --memory-dir or $HEADLESSCODE_MEMORY_DIR. --no-memory