opencode-harness-kit 0.1.0 → 0.2.1

package/GLOBAL.md

@@ -54,7 +54,7 @@ to avoid this.
 
 ## Never
 
-- `git push`, publish, or open a pull request. A human does that.
+- Ask before `git push`, publish, or opening a pull request. Publishing is a human decision.
 - Commit a secret. If you find one, report it; do not move it.
 - Read or write `.env`, `.npmrc`, `.netrc`, `.git-credentials`. The guard
   blocks these and will keep blocking them.

package/README.md

@@ -1,21 +1,34 @@
 # opencode-harness-kit
 
-[![CI](https://github.com/crytobot459/space-harness/actions/workflows/ci.yml/badge.svg)](https://github.com/crytobot459/space-harness/actions/workflows/ci.yml)
+[![CI](https://github.com/pham-canh/space-harness/actions/workflows/ci.yml/badge.svg)](https://github.com/pham-canh/space-harness/actions/workflows/ci.yml)
 [![npm](https://img.shields.io/npm/v/opencode-harness-kit.svg)](https://www.npmjs.com/package/opencode-harness-kit)
 
+<p align="center">
+  <video src="docs/demo/pitch.mp4" width="720" controls></video>
+</p>
+
+*Four minutes forty-seven seconds, recorded live from a real terminal and
+narrated. The take script and the voiceover are in [`video/`](video/), and
+rebuilding it is documented there — including the two mistakes that cost the
+most time, so the next person does not repeat them.*
+
 A harness kit for [OpenCode](https://opencode.ai) that is **model-agnostic by
 construction**. Switching models is a data edit, not a code change.
 
 ```bash
-npx oc-harness init                  # pick a free model, then install + verify
+npx -y opencode-harness-kit init    # pick a free model, then install + verify
 ```
 
+<p align="center">
+  <img src="docs/demo-loop.gif" width="760" alt="oc-harness list showing the model-agnostic profiles, then oc-harness verify reporting a consistent install">
+</p>
+
 then restart opencode. That is the whole quickstart. No npm account needed on
 your side — `npx` only downloads the published kit. Prefer git? The same three
 steps from a checkout (see [Install](#install)):
 
 ```bash
-git clone https://github.com/crytobot459/space-harness.git
+git clone https://github.com/pham-canh/space-harness.git
 ./bin/oc-harness.mjs init
 ```
 
@@ -47,7 +60,7 @@ profiles/space-bunny-free.json     ← the model
 src/                               ← no model name anywhere in here
 ```
 
-Same plugin source, four shipped profiles, four different behaviours. The test
+Same plugin source, five shipped profiles, five different behaviours. The test
 suite asserts that rather than taking it on faith.
 
 ## What the compiler derives
@@ -114,24 +127,81 @@ on install. It is not hidden, it is labelled.
 | profile | context | top effort | retention |
 |---|---|---|---|
 | `space-bunny-free` (default) | 1,048,576 | `max` | zero-retention |
+| `muse-split-free` | 1,048,576 | `max` | **may train** |
 | `x-preview-f-free` | 1,000,000 | `max` | zero-retention |
 | `longcat-2.5-preview-free` | 1,000,000 | — | zero-retention |
 | `deepseek-v4-flash-free` | 200,000 | `max` | zero-retention |
 | `muse-spark-1.3-contributor-free` | 1,048,576 | `xhigh` | **may train** |
 
+### Split profiles
+
+A profile normally puts one model behind every role. `roleModels` is the way to
+split that, for when the best free model and the highest-output model are not the
+same model:
+
+```jsonc
+"model": "space-bunny-free",          // base: every role not named below
+"roleModels": {
+  "plan":     { "model": "muse-spark-1.3-contributor-free", "levels": ["minimal","low","medium","high","xhigh"] },
+  "build":    { "model": "muse-spark-1.3-contributor-free", "levels": ["minimal","low","medium","high","xhigh"] },
+  "locate":   { "model": "space-bunny-free", "levels": ["low","medium","high","xhigh","max"] }
+}
+```
+
+Three things fall out of this that are easy to get wrong, so they are enforced:
+
+- **`levels` is required when the model differs.** Effort is resolved against the
+  levels of the model the role actually runs on. `plan` asking for `max` on a
+  model that stops at `xhigh` lands on `xhigh`, instead of writing a `max` variant
+  the model has no meaning for.
+- **Retention is the least private model in the file, not the base one.** A base
+  of `zero-retention` with a `may-train` role is a `may-train` profile. The
+  generated header prints that, with the model responsible, and so do
+  `oc-harness use` and `oc-harness show`.
+- **Each role's output cap is its own.** A split role inherits the limits of the
+  model it names, so a `build` on a 131,072-token model cannot spend the base's
+  524,288.
+
+`muse-split-free` is the shipped example, and it is deliberately not the default:
+three of its five roles run on Muse Spark, a Meta contributor-tier endpoint whose
+provider may train on your prompts. Install it for public or throwaway
+repositories.
+
+## Requirements
+
+| | |
+|---|---|
+| Node | **22 or newer** — `engines` in package.json, and the CLI uses `node:` prefixed builtins. Node 20 fails at import time, not with a friendly message. |
+| npm | 10 or newer, for `npm ci` when installing from a checkout |
+| OS | macOS, Linux and Windows. No platform checks in the code; the tests run on `ubuntu-latest`. |
+| Shell | none. The CLI never shells out, so no `bash`, `zsh` or `fish` setup is needed. |
+| Network | `list` and `doctor` read models.dev, which is the point of both. Everything else, including `install`, `use` and `verify`, works offline. |
+| Disk | it writes into `~/.config/opencode/` and nothing else. |
+
+`~/.config/opencode/` is your live agent config. `use` and `install` write there,
+`verify` only reads, and `use --dryrun` writes nothing at all. Every write keeps
+a `.bak` next to the file, including under `--yes`.
+
 ## Commands
 
 ```
 oc-harness list       profiles with retention and caveats
-oc-harness use <id>   compile a profile into ~/.config/opencode/opencode.jsonc
 oc-harness show <id>  resolved effort, derived numbers, fallback queue
+oc-harness init       pick a free profile, then install + verify
+oc-harness use <id>   compile a profile into ~/.config/opencode/opencode.jsonc
+oc-harness use <id> --dryrun   print what would be written, write nothing
+oc-harness refresh <id> re-derive a profile against the live model list
 oc-harness install    symlink the plugin, agents, skills and GLOBAL.md
+oc-harness link       just the plugin and instruction file, no agents or skills
 oc-harness verify     check the install against the profile; non-zero on drift
 oc-harness agents     report agents the effort tiering does not reach
 oc-harness doctor     re-check profiles against models.dev and the live config
 oc-harness baseline   measure the current install
 ```
 
+`oc-harness <anything-else>` now exits non-zero with the list of real commands,
+instead of printing help and reporting success.
+
 ### `verify` is the command that matters
 
 A generated config is only half an install. The other half is the instruction
@@ -235,8 +305,10 @@ Your numbers stay on your machine; only the script is in the repo.
 it was handed. It splits on shell separators, honours quotes so that a `;`
 inside a string is an argument rather than a second command, strips `env`,
 `sudo` and `git -C` before matching, and follows `sh -c "..."` into its payload.
-So `cd repo && git push`, `sudo rm -rf /` and `bash -c "git push"` are all
-caught, while `git log --grep push` and `git push origin my-feature-f` are not.
+So `cd repo && rm -rf /`, `sudo rm -rf /` and `bash -c "rm -rf /"` are all
+caught, while `git log --grep push` and `echo push` are not. `git push` is
+not guard-blocked; it falls through to the `*` ask permission so opencode
+prompts for human approval before publishing.
 Writes are scoped to the project directory, with the OS temp directory as the
 one exception GLOBAL.md allows.
 
@@ -290,12 +362,11 @@ pkg install nodejs git
 npx -y opencode-harness-kit init
 ```
 
-Not published to npm yet, or no registry access? The checkout path works
-identically:
+Without registry access, the checkout path works identically:
 
 ```bash
 pkg install nodejs git
-git clone https://github.com/crytobot459/space-harness.git
+git clone https://github.com/pham-canh/space-harness.git
 ./space-harness/bin/oc-harness.mjs init
 ```
 
@@ -417,6 +488,15 @@ Measured results, and the limits they do not cross, are in
 `docs/RESEARCH.md` records where the numbers came from and what was tried and
 rejected.
 
+## Contributing
+
+[`CONTRIBUTING.md`](CONTRIBUTING.md) — the bar is that every change is
+verifiable. Offline tests stay offline, anything that writes to disk needs a
+`--dryrun` and a backup, and a number that moves updates the README in the same
+commit. Bug reports go to [SECURITY.md](SECURITY.md) if they involve a key or
+someone else's config; otherwise open an issue with the command you ran and what
+it printed.
+
 ## License
 
 MIT

package/agents/build.md

@@ -2,7 +2,7 @@
 description: Writes code and verifies it. Use when the plan is settled and files need to change.
 mode: primary
 model: opencode/space-bunny-free
-variant: high
+variant: max
 steps: 40
 permission:
   edit: allow
@@ -50,7 +50,7 @@ You implement one task from a plan. One. Not the whole plan.
   cannot complete the task, say so before writing a partial one.
 - Never widen the change. A drive-by refactor in the same file is a separate
   task; do not report it as part of this one.
-- Never `git push`, never publish, never open a pull request. Committing is
+- Ask before `git push`, publish, or opening a pull request. Committing is
   allowed but only when the task asked for it.
 - Never fabricate a passing result. A failing test reported honestly is worth
   more than a green one you did not get.

package/bin/oc-harness.mjs

@@ -12,7 +12,7 @@
 // The point of the kit: switching models is a data edit, not a code change.
 // Everything model-specific lives in profiles/<id>.json.
 
-import { chmodSync, copyFileSync, mkdirSync, readFileSync, writeFileSync, existsSync, rmSync, symlinkSync, readdirSync, readlinkSync, realpathSync } from "node:fs"
+import { chmodSync, copyFileSync, mkdirSync, readFileSync, writeFileSync, existsSync, rmSync, symlinkSync, readdirSync, readlinkSync, realpathSync, lstatSync } from "node:fs"
 import { homedir } from "node:os"
 import { dirname, join, relative } from "node:path"
 import { execFileSync } from "node:child_process"
@@ -27,9 +27,12 @@ import {
   profilePath,
   pruneThreshold,
   reservedTokens,
+  effectiveRetention,
+  roleSplitModels,
   resolveDefaultVariant,
   resolveRoles,
   ProfileError,
+  parseJsonc,
 } from "../src/profile.js"
 import { generateConfig, globalInstructionsPath } from "../src/config.js"
 import { planEffort } from "../src/effort.js"
@@ -132,7 +135,7 @@ function cmdShow(id) {
   if (!id) fail("usage: oc-harness show <profile>")
   const p = loadProfile(id)
   const plan = planEffort(id)
-  const retention = loadRetention()[`opencode/${p.provider}/${p.model}`]
+  const retention = loadRetention()[`${p.provider}/${p.model}`]
 
   console.log(`\n${bold(p.id)}  ${stamp(p.title ?? "")}\n`)
   const row = (k, v) => console.log(`  ${k.padEnd(20)} ${v}`)
@@ -146,7 +149,20 @@ function cmdShow(id) {
   row("attachment", p.supports?.attachment ? "yes" : stamp("no"))
   row("reserved", reservedTokens(p).toLocaleString("en-US"))
   row("prune at", pruneThreshold(p).toLocaleString("en-US"))
-  row("retention", p.retention === "zero-retention" ? green(p.retention) : yellow(p.retention))
+  // Effective, not the base model's: a split profile's base can be
+  // zero-retention while the roles that matter are not.
+  const eff = effectiveRetention(p)
+  row("retention", eff.retention === "zero-retention" ? green(eff.retention) : yellow(eff.retention))
+  if (eff.retention !== p.retention) {
+    console.log(`  ${"".padEnd(20)} ${stamp(`base ${p.model} is ${p.retention}, but ${eff.worst} makes the whole profile ${eff.retention}`)}`)
+  }
+  if (roleSplitModels(p).length) {
+    console.log(`\n  ${bold("role models")}`)
+    for (const s of roleSplitModels(p)) {
+      const cap = s.limits ? `  out ${s.limits.output.toLocaleString("en-US")}` : ""
+      console.log(`  ${s.role.padEnd(20)} ${s.model}${cap}`)
+    }
+  }
 
   console.log(`\n  ${bold("role effort")}`)
   for (const [role, level] of Object.entries(plan.roles)) {
@@ -358,25 +374,36 @@ async function askChoice(question) {
 function linkOne(src, dest) {
   const parent = realpathSync(dirname(dest))
   const target = relative(parent, realpathSync(src))
-  let stale = false
+
+  let isLink = false
+  let ours = false
   try {
-    stale = readlinkSync(dest) !== target
+    isLink = lstatSync(dest).isSymbolicLink()
+    ours = isLink && readlinkSync(dest) === target
   } catch {
-    stale = true
+    // Nothing at `dest` yet.
   }
-  if (stale) {
-    try {
-      rmSync(dest, { force: true, recursive: false })
-    } catch {
-      // Nothing to remove, or it is a real directory we must not delete.
+  if (ours) return
+
+  if (existsSync(dest)) {
+    // Replace only what this installer put there. The old code treated "not
+    // already our link" as "safe to delete" and called `rmSync` with
+    // `recursive: false`, which succeeds on a plain file: a hand-written
+    // agent was deleted and replaced by a symlink without a word. A real
+    // directory is left alone too -- unlinking it is not this installer's
+    // call to make.
+    if (!isLink) {
+      const kind = lstatSync(dest).isDirectory() ? "directory" : "file"
+      console.log(`  ${yellow("!")} ${stamp(`left alone: ${dest} is a real ${kind} and was not ours to replace`)}`)
+      return
     }
+    rmSync(dest, { force: true })
   }
-  if (!existsSync(dest)) {
-    try {
-      symlinkSync(target, dest)
-    } catch (err) {
-      console.log(`  ${yellow("!")} ${stamp(`could not link ${dest}: ${err.message}`)}`)
-    }
+
+  try {
+    symlinkSync(target, dest)
+  } catch (err) {
+    console.log(`  ${yellow("!")} ${stamp(`could not link ${dest}: ${err.message}`)}`)
   }
 }
 
@@ -405,9 +432,7 @@ function cmdVerify() {
   //    hand-edit anywhere in the model/variant/permission surface shows up
   //    here, which is the check that would have caught `build` being moved onto
   //    a may-train model.
-  const actual = existsSync(CONFIG_FILE)
-    ? stripComments(readFileSync(CONFIG_FILE, "utf8"))
-    : null
+  const actual = existsSync(CONFIG_FILE) ? readConfigJson() : null
   if (!actual) {
     problems.push(`${CONFIG_FILE} does not exist. Run: oc-harness use ${p.id}`)
   } else {
@@ -563,7 +588,14 @@ function configDrift(p, actual) {
       out.push(`agent.${role} missing`)
       continue
     }
-    if (agent.model !== want) out.push(`agent.${role}.model ${agent.model} -> ${want}`)
+    // A role's expected model is the one roleModels declares for it, falling
+    // back to the base. Comparing every agent against `want` would report
+    // drift on every split role forever, which trains the user to ignore the
+    // one command that exists to catch a moved `build`.
+    const wantRole = `${p.provider}/${p.roleModels?.[role]?.model ?? p.model}`
+    if (agent.model !== wantRole) {
+      out.push(`agent.${role}.model ${agent.model} -> ${wantRole}`)
+    }
     if (agent.variant !== (level ?? undefined)) {
       out.push(`agent.${role}.variant ${agent.variant ?? "(none)"} -> ${level ?? "(none)"}`)
     }
@@ -609,10 +641,7 @@ async function cmdUse(id) {
     console.error(msg)
   }
 
-  let base = {}
-  if (existsSync(CONFIG_FILE)) {
-    base = stripComments(readFileSync(CONFIG_FILE, "utf8"))
-  }
+  const base = readConfigJson()
 
   const out = generateConfig(p, { base, comments: true })
   if (has("--print")) {
@@ -620,11 +649,22 @@ async function cmdUse(id) {
     return
   }
 
+  // A flag that is accepted and then ignored is worse than one that errors:
+  // `--dryrun` used to write the real config anyway.
+  if (has("--dryrun")) {
+    console.log(`\n  ${stamp("dry run, nothing written")}`)
+    console.log(`  ${stamp(`would write ${CONFIG_FILE}`)}`)
+    summary(p)
+    return
+  }
+
   mkdirSync(dirname(CONFIG_FILE), { recursive: true })
-  if (existsSync(CONFIG_FILE) && !has("--yes")) {
-    // The config is the user's file once written. A backup is made either
-    // way, but overwriting it without asking is how a hand-tuned setup dies.
-    if (interactive() && !(await confirm(`overwrite ${CONFIG_FILE}? (a .bak backup is kept)`))) {
+  if (existsSync(CONFIG_FILE)) {
+// The config is the user's file once written. The backup is unconditional:
+    // `--yes` skips the question, it does not skip the safety net. Gating the
+    // backup on `--yes` meant the one path that overwrites without asking was
+    // also the one path with nothing to fall back to.
+    if (!has("--yes") && interactive() && !(await confirm(`overwrite ${CONFIG_FILE}? (a .bak backup is kept)`))) {
       fail("left the existing config alone")
     }
     writeFileSync(`${CONFIG_FILE}.bak`, readFileSync(CONFIG_FILE))
@@ -632,23 +672,48 @@ async function cmdUse(id) {
   }
   writeFileSync(CONFIG_FILE, out)
   console.log(`\n  ${green("✓")} wrote ${CONFIG_FILE}`)
+  summary(p)
+  console.log(`\n  ${stamp("verify with: opencode debug config")}\n`)
+}
+
+/** What the profile puts behind the user, including a split and real retention. */
+function summary(p) {
+  const eff = effectiveRetention(p)
+  const splits = roleSplitModels(p)
   console.log(`  ${stamp(`model      ${p.provider}/${p.model}`)}`)
   console.log(`  ${stamp(`effort     ${resolveDefaultVariant(p) ?? "(not tunable)"}`)}`)
   console.log(`  ${stamp(`roles      ${Object.entries(resolveRoles(p)).map(([k, v]) => `${k}=${v ?? "-"}`).join(" ")}`)}`)
+  if (splits.length) {
+    // Without this the split is invisible until someone opens the file, which
+    // is too late to be told which roles send prompts to a provider that trains.
+    console.log(`  ${stamp(`split      ${splits.map((s) => `${s.role}=${s.model}`).join(" ")}`)}`)
+    console.log(`  ${stamp(`           ${splits.map((s) => s.limits ? `${s.role} out ${s.limits.output.toLocaleString("en-US")}` : null).filter(Boolean).join("  ")}`)}`)
+  }
+  console.log(`  ${stamp(`retention  ${eff.retention}${eff.retention === "may-train" ? "  <-- prompts may train a model" : ""}`)}`)
+  if (splits.length && eff.retention !== p.retention) {
+    console.log(`  ${stamp(`           base ${p.model} is ${p.retention}, but ${eff.worst} makes the whole profile ${eff.retention}`)}`)
+  }
   console.log(`  ${stamp(`reserved   ${reservedTokens(p).toLocaleString("en-US")} tokens`)}`)
-  console.log(`\n  ${stamp("verify with: opencode debug config")}\n`)
 }
 
-/** opencode.jsonc is JSONC, so strip comments before parsing. */
-function stripComments(text) {
-  const without = text
-    .replace(/^\s*\/\/.*$/gm, "")
-    .replace(/\/\*[\s\S]*?\*\//g, "")
-  try {
-    return JSON.parse(without)
-  } catch {
-    return {}
+/**
+ * Read the installed config as JSONC.
+ *
+ * Strict on purpose. Returning `{}` for a config we could not parse made
+ * `oc-harness use` merge onto an empty base and write the user's own settings
+ * back out deleted, with no message. Refusing is the only safe answer here.
+ */
+function readConfigJson() {
+  if (!existsSync(CONFIG_FILE)) return {}
+  const parsed = parseJsonc(readFileSync(CONFIG_FILE, "utf8"))
+  if (!parsed) {
+    fail(
+      `${CONFIG_FILE} is not valid JSONC and was left untouched.\n` +
+        `  Refusing to regenerate: the existing settings cannot be read, so merging\n` +
+        `  onto them would silently drop them. Fix the file by hand, then re-run.`,
+    )
   }
+  return parsed
 }
 
 /** Re-fetch models.dev and report which profiles have drifted from it. */
@@ -790,7 +855,7 @@ async function cmdDoctor() {
       problems++
       console.log(`  ${yellow("!")} ${id.padEnd(32)} ${yellow("drift")}`)
       for (const i of issues) console.log(`      ${i}`)
-      console.log(`      ${stamp(`fix: oc-harness use ${id} --refresh`)}`)
+      console.log(`      ${stamp(`fix: oc-harness refresh ${id}`)}`)
     }
   }
   console.log(
@@ -810,7 +875,7 @@ async function cmdDoctor() {
     console.log(`  ${yellow("!")} ${stamp("no profile recorded in the config. Run: oc-harness use <profile>")}\n`)
     return
   }
-  const installed = existsSync(CONFIG_FILE) ? stripComments(readFileSync(CONFIG_FILE, "utf8")) : null
+  const installed = existsSync(CONFIG_FILE) ? readConfigJson() : null
   if (!installed) {
     console.log(`  ${yellow("!")} ${stamp(`${CONFIG_FILE} missing. Run: oc-harness use ${activeId}`)}\n`)
     return
@@ -860,7 +925,7 @@ function cmdAgents() {
     fail("no profile is installed. Run: oc-harness use <profile>")
   }
 
-  const config = existsSync(CONFIG_FILE) ? stripComments(readFileSync(CONFIG_FILE, "utf8")) : {}
+  const config = readConfigJson()
   const configured = config.model ?? "(none)"
   const wantModel = `${current.provider}/${current.model}`
 
@@ -912,6 +977,32 @@ function fail(msg) {
   process.exit(1)
 }
 
+/**
+ * Drive the task file to done without a human at the keyboard.
+ *
+ * Every stop condition is printed, and the exit code says which kind of stop it
+ * was: 0 finished, 1 nothing to plan, 2 needs a person, 3 the agent died,
+ * 4 verify failed. A caller can therefore tell "done" from "gave up" without
+ * reading the transcript.
+ */
+async function cmdRun() {
+  const { runLoop } = await import("../src/driver/run.mjs")
+
+  const result = runLoop({
+    cwd: process.cwd(),
+    role: opt("--role") ?? "build",
+    maxRounds: Number(opt("--max-rounds") ?? 20),
+    budgetMinutes: Number(opt("--budget") ?? 120),
+  })
+
+  const s = result.summary
+  console.log(`\n  ${bold(result.code === 0 ? "done" : "stopped")} ${result.reason}`)
+  if (s) {
+    console.log(`  ${stamp(`${s.done} done, ${s.blocked} blocked, ${s.pending} pending`)}`)
+  }
+  if (result.code !== 0) process.exitCode = result.code
+}
+
 try {
   switch (cmd) {
     case "list":
@@ -947,6 +1038,9 @@ try {
     case "baseline":
       await import("./baseline.mjs")
       break
+    case "run":
+      await cmdRun()
+      break
     case "help":
     default:
       console.log(`
@@ -963,13 +1057,25 @@ try {
     agents        report agents the effort tiering does not reach
     doctor    re-check profiles against models.dev and the installed config
     baseline  measure the current opencode install
+    run       drive .space/tasks.md to done with the build agent
 
   options
     --print        write the config to stdout instead of disk
     --yes          accept a retention warning, back up the old config
     --retention    filter list by retention class
     --free         list free profiles only
+
+  run options
+    --role <agent>  which agent does the work (default: build)
+    --max-rounds N  stop after N tasks (default: 20)
+    --budget MIN    stop after MIN minutes (default: 120)
 `)
+    // An unknown command is a typo, not a request for help. Without this
+    // `oc-harness tsl` prints help and exits 0, so a script calling it never
+    // finds out that nothing ran.
+    if (cmd !== "help" && cmd !== "--help" && cmd !== "-h") {
+      fail(`unknown command: ${cmd}\n  run 'oc-harness help' for the list`)
+    }
   }
 } catch (err) {
   if (err instanceof ProfileError) fail(err.message)