@ouro.bot/cli 0.1.0-alpha.982 → 0.1.0-alpha.984

package/README.md

@@ -1,6 +1,6 @@
 <!-- ourostack:brand:start -->
 <p align="center">
-  <a href="https://github.com/ourostack"><img src="https://raw.githubusercontent.com/ourostack/.github/main/brand/ouro.png" alt="Ouro" width="160" height="160"></a>
+  <a href="https://github.com/ourostack"><img src="https://raw.githubusercontent.com/ourostack/.github/main/brand/headers/ouroboros.webp" alt="Ouro outside its round-doored home in a flowering hillside, with letters and speech bubbles floating out of the windows and a telephone and mailbox by the door" width="100%"></a>
 </p>
 <!-- ourostack:brand:end -->
 

package/changelog.json

@@ -1,6 +1,18 @@
 {
   "_note": "This changelog is maintained as part of the PR/version-bump workflow. Agent-curated, not auto-generated. Agents read this file directly via read_file to understand what changed between versions.",
   "versions": [
+    {
+      "version": "0.1.0-alpha.984",
+      "changes": [
+        "The Butler learns house rules from anyone in the house, straight into TACIT."
+      ]
+    },
+    {
+      "version": "0.1.0-alpha.983",
+      "changes": [
+        "The Butler's psyche is his own: he learns into TACIT, his household can change the rest in a one-to-one chat, and his product rules ship separately. Reading his household's names now leaves a trace in the logs."
+      ]
+    },
     {
       "version": "0.1.0-alpha.982",
       "changes": [

package/deploy/unraid/README.txt

@@ -6,7 +6,7 @@ Root authority lifecycle
 
 The existing DockerMan transaction owns authority-install, authority-activate, authority-retire and authority-restore. A complete exact-image package, including dist and production dependencies, must remain at its package-relative paths; extracting only the old five deployment assets cannot execute this lifecycle. The reviewed root-only inputs are /mnt/user/appdata/ouro-authority/incoming-package, package-manifest.json, request.json and incoming-token. They are not a new resident credential policy. The human rotates the existing bot token directly into root custody; the root owner proves the previous token returns 401 and the fresh token belongs to the same numeric bot before continuing.
 
-Psyche mount retired: alpha.978 and earlier mounted the bundle psyche folder read-only over the bundle (six mounts). The upgrade audits still accept that shape, so the running alpha.978 resident passes them; the upgrade then recreates the resident with five mounts (no separate psyche mount) and removes the psyche Config (Target /home/ouro/AgentBundles/sanctuary.ouro/psyche) from the installed DockerMan template /boot/config/plugins/dockerMan/templates-user/my-ouro-butler.xml, so a DockerMan Apply cannot add the read-only mount back. Rolling the upgrade back recreates the predecessor with the read-only psyche mount it expects. Psyche files stay owned by uid 10001 at mode 0600 because alpha.978's boot check requires exactly that after a rollback.
+Psyche mount retired: alpha.978 and earlier mounted the bundle psyche folder read-only over the bundle (six mounts). The upgrade audits still accept that shape, so the running alpha.978 resident passes them; the upgrade then recreates the resident with five mounts (no separate psyche mount) and removes the psyche Config (Target /home/ouro/AgentBundles/sanctuary.ouro/psyche) from the installed DockerMan template /boot/config/plugins/dockerMan/templates-user/my-ouro-butler.xml, so a DockerMan Apply cannot add the read-only mount back. Rolling an upgrade back to alpha.978 or earlier recreates the predecessor with the read-only psyche mount it expects; a rollback to alpha.979 or later recreates it with no psyche mount. Psyche files stay owned by uid 10001, not writable by group or others, with no symlinks (verify checks exactly that); the upgrade still resets them to 0600, because exact 0600 matters only for a rollback to alpha.978 or earlier, whose boot check requires it. From M1b the Butler writes its own TACIT and a family member in a one-to-one chat (Telegram, BlueBubbles, or Teams) can change the rest of its psyche; every gated upgrade plants psyche/.release-marker and reports psyche-survives-release (and psyche-survives-rollback after a gate failure), and a drill (b) style downgrade to alpha.979 runs with --no-gate.
 
 Operator trust directory. Delegated-command and escalation grants live on the host in /mnt/user/appdata/ouro-butler/trust/sanctuary, owned by root:root, mode 0755, with every file in it root-owned and mode 0644, and the directory is outside the bundle so the upgrade's recursive chown cannot reach it. The container sees it read-only at /etc/ouro/trust/sanctuary; the Butler honours a grant only when the file and every directory above it up to / are real, root-owned and closed to group and other writes. Create it once as root before the first create: `install -d -o root -g root -m 0755 /mnt/user/appdata/ouro-butler /mnt/user/appdata/ouro-butler/trust /mnt/user/appdata/ouro-butler/trust/sanctuary` (this also makes /mnt/user/appdata/ouro-butler itself root-owned 0755, which the replay gate needs for its host folder below; the agent and runtime folders inside it keep their own owners). Manage grants on the host as root with a one-off container that mounts the directory read-write (the resident's mount is read-only): `docker run --rm --user 0:0 --entrypoint node --pull=never --network none -v /mnt/user/appdata/ouro-butler/trust/sanctuary:/etc/ouro/trust/sanctuary -v /mnt/user/appdata/ouro-butler/agent/sanctuary.ouro:/home/ouro/AgentBundles/sanctuary.ouro:ro <butler image> /opt/ouro/dist/heart/daemon/ouro-entry.js a2a delegated-commands grant --agent sanctuary --friend <friend id> --did <did:key> [--expires <ISO date>] [--source <text>]`, and likewise `a2a delegated-commands revoke|list` and `a2a escalation grant --friend <id> --did <did:key>|revoke|list`. Setting OURO_OPERATOR_TRUST_DIR replaces the trust root (/etc/ouro/trust) and keeps the per-agent folder, so the files sit in <dir>/<agent>/. Grant commands take an exclusive .lock file in the directory (the lock records the writer's pid and host; one older than two minutes is taken over only when that pid is no longer running on the same host, and one written on another host only after an hour; remove a stuck one by hand) and flush the directory after each rename. A one-off container cannot read the agent's own DID, so grant then compares --did with every friend record and refuses a DID a second record also carries, and prints a note saying so; revoke works by friend id even when the friend record is gone. Revoke never writes the friend record (it runs as root); when the record still carries a legacy delegationGrant it prints a REQUIRED FOLLOW-UP docker exec -u 10001 line that clears the legacy delegationGrant field as the resident user (friend records are never written as root). The replay gate grants its replay principal and escalation peer with an expiry equal to the replay window and removes them when it closes; the replay gate keeps a root-owned list of its replay peers in the trust directory (replay-identities.json), the Butler refuses any grant without an expiry for a friend on that list (the grant's source text and the bundle's own replay registry decide nothing), and the grant commands refuse a no-expiry grant, and any escalation grant, for a listed friend. Replay seeds and the provisioning record live outside the bundle, in the root-owned host folder /mnt/user/appdata/ouro-butler/replay-client (the gate's --client-dir overrides it), created by provision as root 0700 under a parent that is root-owned and closed to group and other writes (by default only that parent, /mnt/user/appdata/ouro-butler, is checked, because Unraid's shared /mnt/user/appdata is usually nobody:users 0777); the gate refuses a folder, a parent or a seed that root did not make, and never takes over one the Butler's user could have written. Only root-owned state may supply replay friend ids and DIDs: that folder and the trust directory's replay-identities.json. Nothing in the bundle's state/replay-client or state/replay does, and a provision.json found in the bundle is ignored (its friend ids are only printed, as unverified). A run copies the seeds into a root-only directory it makes inside the container with mktemp -d, drives the CLI as root from there, and deletes it when the window closes. An existing state/replay in the bundle that root does not own is moved aside as replay.untrusted-<time> and made fresh. One-time step on the upgrade that moves the replay seeds out of the bundle: run `node sanctuary-replay-gate.mjs provision --rotate-replay-identities` as root. Earlier replay seeds were written by the Butler's user and must be treated as known to it, so this mints three new seeds (root-owned 0600, in the host folder above), moves any old seed in that folder aside as *.retired-<time>, removes the retired peers' trusted grants, marks their friend records revoked, demotes them to stranger and drops their DID pin and A2A identity (the last two as the resident user, never as root), onboards the new peers and rewrites state/replay/identities.json from the root-owned list; the old friend ids stay in that registry so they stay blocked from owner-policy writes. The peers it retires are the ones the root-owned record and replay list name. The earlier bundle copy of provision.json is not trusted, so the first rotation prints the friend ids it names as unverified; for each one you have checked is a replay peer, add `--retire-friend <id>` (repeatable). Always pass the DID the holder really signs with (`ouro a2a identity --json` on the holder's side); the Butler refuses a message signed by any other DID. First rollout of the trust mount (alpha.887): the live container then had no trust mount, so the first update failed the mount audit exactly as the psyche rollout did; the fix was to recreate the resident once by hand with the runbook's canonical-gateway create and rerun the update. If a resident without the trust mount is ever found again, recreate the resident once by hand with the five-mount create and rerun the update.
 

package/deploy/unraid/sanctuary-butler-upgrade.mjs

@@ -47,8 +47,8 @@
 //        sanctuary-butler-upgrade.mjs verify [--gate]
 
 import { execFileSync, spawn, spawnSync } from "node:child_process"
-import { createHash } from "node:crypto"
-import { mkdtempSync, rmSync, writeFileSync, readFileSync, existsSync, statSync, lstatSync, chmodSync, realpathSync, readdirSync } from "node:fs"
+import { createHash, randomBytes } from "node:crypto"
+import { mkdtempSync, rmSync, writeFileSync, readFileSync, existsSync, statSync, lstatSync, chmodSync, realpathSync, readdirSync, openSync, closeSync, fchownSync, fchmodSync, fstatSync, readSync, renameSync, unlinkSync, constants as fsConstants } from "node:fs"
 import { fileURLToPath } from "node:url"
 import { createRequire } from "node:module"
 import { tmpdir } from "node:os"
@@ -598,21 +598,128 @@ function restorePsycheRoot() {
   if (existsSync(dir)) restorePsycheModes(dir)
 }
 
-/** What is wrong with the psyche folder as the resident's boot check needs it: files owned by the resident at exactly 0600, directories not writable by group or others, no symlinks. Empty when it is sound. */
+/** What is wrong with the psyche folder: anything not owned by the resident, writable by group or others, or a symlink. A file the Butler creates may be 0644. Empty when it is sound. */
 export function psycheProblems(dir, { lstat = lstatSync, readdir = readdirSync } = {}) {
   const problems = []
   const walk = (current) => {
     const stat = lstat(current)
     if (stat.isSymbolicLink()) { problems.push(`${current} is a symlink`); return }
     if (stat.uid !== 10001) problems.push(`${current} is owned by uid ${stat.uid}, not the resident (10001)`)
-    if (stat.isDirectory()) {
-      if ((stat.mode & 0o022) !== 0) problems.push(`${current} is writable by group or others (mode ${(stat.mode & 0o777).toString(8)})`)
-      for (const name of readdir(current)) walk(`${current}/${name}`)
-    } else if ((stat.mode & 0o777) !== 0o600) problems.push(`${current} is mode ${(stat.mode & 0o777).toString(8)}, not 600`)
+    if ((stat.mode & 0o022) !== 0) problems.push(`${current} is writable by group or others (mode ${(stat.mode & 0o777).toString(8)})`)
+    if (stat.isDirectory()) for (const name of readdir(current)) walk(`${current}/${name}`)
   }
   walk(dir)
   return problems
 }
+// A gated upgrade plants a marker in the psyche and hashes the four identity files, then proves both survived the release
+// and any rollback. TACIT is the Butler's own and may change, so only that it is there is checked.
+// Root works inside a folder the Butler's user owns, which could rename psyche/ and put a symlink in its place at any moment.
+// So every operation goes through one descriptor held on the real psyche/ (/proc/self/fd/<fd>/<name> on the Linux host),
+// every read opens a regular file without following links or blocking, and before each write, rename or unlink the path
+// psyche/ must still be that same directory.
+export const PSYCHE_IDENTITY_FILES = ["SOUL.md", "IDENTITY.md", "LORE.md", "ASPIRATIONS.md"]
+const PSYCHE_FILES = [...PSYCHE_IDENTITY_FILES, "TACIT.md"]
+const PSYCHE_MARKER = ".release-marker"
+const PSYCHE_READ_LIMIT = 1024 * 1024
+export const psycheFdPath = (dirfd, name) => `/proc/self/fd/${dirfd}/${name}`
+
+/** Read at most `limit` bytes from an open file; more than that is a problem, even if the file grew after it was checked. */
+export function readPsycheBytes(fd, limit = PSYCHE_READ_LIMIT) {
+  const buffer = Buffer.alloc(limit + 1)
+  let length = 0
+  for (let read; length <= limit && (read = readSync(fd, buffer, length, limit + 1 - length, null)) > 0;) length += read
+  return length > limit ? { problem: "is larger than 1 MiB" } : { bytes: buffer.subarray(0, length) }
+}
+
+function readPsycheFile(dirfd, name, at) {
+  let fd
+  try { fd = openSync(at(dirfd, name), fsConstants.O_RDONLY | fsConstants.O_NOFOLLOW | fsConstants.O_NONBLOCK) } catch (e) {
+    const why = e.code === "ENOENT" ? "is missing" : e.code === "ELOOP" ? "is a symlink" : `could not be opened (${e.code ?? e.message})`
+    return { problem: `psyche/${name} ${why}` }
+  }
+  try {
+    if (!fstatSync(fd).isFile()) return { problem: `psyche/${name} is not a regular file` }
+    const read = readPsycheBytes(fd)
+    return read.problem ? { problem: `psyche/${name} ${read.problem}` } : read
+  } finally { closeSync(fd) }
+}
+
+/** Run `work` with a descriptor on the real psyche/ and a guard that throws if the path psyche/ is no longer that directory. */
+function withPsycheDir(dir, work) {
+  let dirfd
+  try { dirfd = openSync(dir, fsConstants.O_RDONLY | fsConstants.O_DIRECTORY | fsConstants.O_NOFOLLOW) } catch { throw new Error(`${dir} is not a real directory`) }
+  try {
+    const held = fstatSync(dirfd)
+    const guard = () => {
+      let now = null
+      try { now = lstatSync(dir) } catch { /* gone counts as replaced */ }
+      if (!now?.isDirectory() || now.dev !== held.dev || now.ino !== held.ino) throw new Error(`${dir} was replaced while root was working in it; refusing`)
+    }
+    return work(dirfd, guard)
+  } finally { closeSync(dirfd) }
+}
+
+const hashOf = (bytes) => createHash("sha256").update(bytes).digest("hex")
+
+/** Write psyche/.release-marker (0600, resident-owned, a random token) and hash the identity files. Refuses unless all five psyche files are regular files. */
+export function plantPsycheMarker(dir, { uid, gid }, { at = psycheFdPath } = {}) {
+  return withPsycheDir(dir, (dirfd, guard) => {
+    const read = Object.fromEntries(PSYCHE_FILES.map((name) => [name, readPsycheFile(dirfd, name, at)]))
+    for (const name of PSYCHE_FILES) if (read[name].problem) throw new Error(read[name].problem)
+    const identity = Object.fromEntries(PSYCHE_IDENTITY_FILES.map((name) => [name, hashOf(read[name].bytes)]))
+    const token = randomBytes(16).toString("hex")
+    const staged = `${PSYCHE_MARKER}.${token}`
+    guard()
+    const fd = openSync(at(dirfd, staged), fsConstants.O_WRONLY | fsConstants.O_CREAT | fsConstants.O_EXCL | fsConstants.O_NOFOLLOW, 0o600)
+    try {
+      fchownSync(fd, uid, gid)
+      fchmodSync(fd, 0o600)
+      writeFileSync(fd, token)
+    } finally { closeSync(fd) }
+    try { guard() } catch (e) { try { unlinkSync(at(dirfd, staged)) } catch { /* the guard's error is the one to report */ } throw e }
+    renameSync(at(dirfd, staged), at(dirfd, PSYCHE_MARKER))
+    return { token, identity }
+  })
+}
+
+/** Null when the marker holds the token, all five psyche files are regular files and the four identity hashes match; otherwise the first problem. */
+export function psycheSurvival(dir, planted, { at = psycheFdPath } = {}) {
+  try {
+    return withPsycheDir(dir, (dirfd) => {
+      const marker = readPsycheFile(dirfd, PSYCHE_MARKER, at)
+      if (marker.problem) return marker.problem
+      if (marker.bytes.toString("utf8") !== planted.token) return `psyche/${PSYCHE_MARKER} does not hold this upgrade's token`
+      const read = PSYCHE_FILES.map((name) => [name, readPsycheFile(dirfd, name, at)])
+      const unreadable = read.find(([, file]) => file.problem)
+      if (unreadable) return unreadable[1].problem
+      const changed = read.find(([name, file]) => name in planted.identity && hashOf(file.bytes) !== planted.identity[name])
+      return changed ? `psyche/${changed[0]} changed` : null
+    })
+  } catch (e) { return e.message }
+}
+
+/** Remove the marker only while it holds this upgrade's token, reading and unlinking through the same descriptor. Returns why it was left, or null. */
+export function removePsycheMarker(dir, token, { at = psycheFdPath } = {}) {
+  try {
+    return withPsycheDir(dir, (dirfd, guard) => {
+      const marker = readPsycheFile(dirfd, PSYCHE_MARKER, at)
+      if (marker.problem) return marker.problem
+      if (marker.bytes.toString("utf8") !== token) return `psyche/${PSYCHE_MARKER} does not hold this upgrade's token`
+      guard()
+      unlinkSync(at(dirfd, PSYCHE_MARKER))
+      return null
+    })
+  } catch (e) { return e.message }
+}
+
+/** Print the survival check as a gate case line (the shape butler-ship's parseCases reports) and return the problem, if any. */
+export function reportPsycheSurvival(id, dir, planted, log = console.log, options = {}) {
+  const problem = psycheSurvival(dir, planted, options)
+  const check = { name: "the psyche marker and all five psyche files survived, with the four identity files byte-identical", ok: problem === null, ...(problem ? { detail: problem } : {}) }
+  log(JSON.stringify({ id, status: problem ? "fail" : "pass", checks: [check] }))
+  return problem
+}
+
 /**
  * What is wrong with the resident's psyche mount, from `docker inspect` Mounts, or null for a supported shape: no separate psyche
  * mount (current releases), or a read-only one (alpha.978 and earlier, which a rollback recreates).
@@ -1236,6 +1343,12 @@ async function upgrade(version, rehearse, { noGate = false, plant } = {}) {
   if (!telegram.ok) fail(`api.telegram.org is not reachable from this host (${telegram.detail}); the new gateway could not pass its Telegram identity check. Nothing was paused or changed; fix the network and re-run.`)
   if (telegram.skipped) console.log("  note: resuming a pending upgrade; Telegram reachability not re-checked")
   else ok(`api.telegram.org reachable (${telegram.detail})`)
+  const psycheDir = `${BUNDLE}/psyche`
+  let planted = null
+  if (gated) {
+    try { planted = plantPsycheMarker(psycheDir, { uid: 10001, gid: 10001 }) } catch (e) { fail(`could not plant the psyche marker before the gated upgrade; nothing was paused or changed: ${e.message}`) }
+    ok("psyche marker planted and the four identity files hashed")
+  }
   pauseSupervision()
   let failure = null
   try {
@@ -1264,36 +1377,45 @@ async function upgrade(version, rehearse, { noGate = false, plant } = {}) {
   if (gated && !failure) {
     say(`replay gate${plant ? ` (planted failure: ${plant})` : ""}`)
     const stillHeld = () => heldUpgradeProblem({ journalExists: existsSync(UPGRADE_JOURNAL), liveImage: butlerState().split(" ")[0], expectedImage: image(version) })
-    const outcome = completeGatedUpgrade({
-      runGate: () => {
-        if (preserved.length) return { ok: false, detail: preserved.join("; ") }
-        if (!waitForButler(image(version), 300)) return { ok: false, detail: `the new Butler did not become healthy on ${image(version)}` }
-        const result = runReplayGate(plant)
-        if (!result.ok) return result
-        const problem = stillHeld()
-        return problem ? { ok: false, detail: problem } : result
-      },
-      commit: () => {
-        pauseSupervision()
-        try {
+    let outcome
+    try {
+      outcome = completeGatedUpgrade({
+        runGate: () => {
+          if (preserved.length) return { ok: false, detail: preserved.join("; ") }
+          if (!waitForButler(image(version), 300)) return { ok: false, detail: `the new Butler did not become healthy on ${image(version)}` }
+          const lost = reportPsycheSurvival("psyche-survives-release", psycheDir, planted)
+          if (lost) return { ok: false, detail: `the psyche did not survive the release: ${lost}` }
+          const result = runReplayGate(plant)
+          if (!result.ok) return result
           const problem = stillHeld()
-          if (problem) throw new Error(`refusing to commit: ${problem}`)
-          ok(`commit: ${sh("/usr/local/bin/node", [lifecycle, "upgrade", id, image(version)], { stdio: ["ignore", "pipe", "pipe"] }).trim()}`)
-        } finally { resumeSupervision() }
-      },
-      rollback: () => {
-        pauseSupervision()
-        try { ok(`rollback: ${sh("/usr/local/bin/node", [lifecycle, "upgrade-rollback"], { stdio: ["ignore", "pipe", "pipe"] }).trim()}`) }
-        catch (e) { fail(ROLLBACK_FAILED(e, lifecycle)) }
-        resumeSupervision()
-      },
-      confirmRollback: () => {
-        if (!waitForButler(priorImage, 300)) fail(`rolled back, but ${priorImage} is not running healthy; inspect \`docker logs ${CONTAINER}\``)
-        return priorImage
-      },
-      pin: () => pinTemplate(version),
-      prune: () => pruneButlerImages(),
-    })
+          return problem ? { ok: false, detail: problem } : result
+        },
+        commit: () => {
+          pauseSupervision()
+          try {
+            const problem = stillHeld()
+            if (problem) throw new Error(`refusing to commit: ${problem}`)
+            ok(`commit: ${sh("/usr/local/bin/node", [lifecycle, "upgrade", id, image(version)], { stdio: ["ignore", "pipe", "pipe"] }).trim()}`)
+          } finally { resumeSupervision() }
+        },
+        rollback: () => {
+          pauseSupervision()
+          try { ok(`rollback: ${sh("/usr/local/bin/node", [lifecycle, "upgrade-rollback"], { stdio: ["ignore", "pipe", "pipe"] }).trim()}`) }
+          catch (e) { fail(ROLLBACK_FAILED(e, lifecycle)) }
+          resumeSupervision()
+        },
+        confirmRollback: () => {
+          if (!waitForButler(priorImage, 300)) fail(`rolled back, but ${priorImage} is not running healthy; inspect \`docker logs ${CONTAINER}\``)
+          reportPsycheSurvival("psyche-survives-rollback", psycheDir, planted)
+          return priorImage
+        },
+        pin: () => pinTemplate(version),
+        prune: () => pruneButlerImages(),
+      })
+    } finally {
+      const left = removePsycheMarker(psycheDir, planted.token)
+      if (left) warn(`the psyche marker was left in place: ${left}`)
+    }
     if (!outcome.passed) process.exit(1)
     console.log(`\nUPGRADE done — ${version} live and gate-verified. Run verify.`)
     return
@@ -1324,7 +1446,7 @@ function verify(withGate = false) {
   const psycheDir = `${BUNDLE}/psyche`
   if (existsSync(psycheDir)) {
     const psycheIssues = psycheProblems(psycheDir)
-    psycheIssues.length === 0 ? ok("psyche files are resident-owned 0600") : bad(`psyche files do not match what the resident boot check requires: ${psycheIssues.slice(0, 3).join("; ")}`)
+    psycheIssues.length === 0 ? ok("psyche files are resident-owned and writable") : bad(`psyche files must be owned by the resident (uid 10001), not writable by group or others, and not symlinks: ${psycheIssues.slice(0, 3).join("; ")}`)
   } else bad("psyche folder missing from the bundle")
   ok(containerExists("jellyfin") ? `jellyfin ${docker(["inspect", "jellyfin", "--format", "{{.State.Status}} restarts={{.RestartCount}}"]).trim()}` : "no jellyfin container")
   if (withGate) {