@awebai/oats 0.22.0 → 0.22.1

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@awebai/oats",
-  "version": "0.22.0",
+  "version": "0.22.1",
   "description": "OATS (Open Agent Team Specification) — durable souls, disposable instances, targetable capability packages, and the runtime-neutral oats CLI/kernel.",
   "keywords": [
     "agents",

package/packages/record/README.md

@@ -6,12 +6,17 @@ contract is [`docs/turn-record-sot.md`](docs/turn-record-sot.md) in this
 package; the conformance vectors under `test/vectors/` pin the format
 (`node test/vectors/validate.mjs`, dependency-free).
 
-Everything is a signed turn in an append-only record. This package is the
-core: gathering every conversation, keeping it durably, and finding it
-again. The experimental tools that select and synthesize over the record
-(dress, spawn, segments, mind) live in `packages/experimental` of the oats
-repo and run as `oats experimental <cmd>`; they are deliberately not part
-of this package.
+Turns in this append-only record are content-addressed. Native session turns
+are not signed. Projected aweb mail and chat keep their original message
+signatures verbatim.
+
+After `turn-record setup` runs on a machine, this package gathers Claude Code,
+Pi, and Codex transcripts plus aw client logs. It skips sources matched by the
+local record's ignore list. The package keeps captured conversations durably
+and finds them again. The experimental tools that select and synthesize over
+the record (dress, spawn, segments, mind) live in `packages/experimental` of
+the oats repo and run as `oats experimental <cmd>`. They are deliberately not
+part of this package.
 
 - **`lib/canonical.mjs`** — canonical JSON (integers only in the core),
   `t1:` content ids, did:key Ed25519 verification. Byte-compatible with awid
@@ -126,16 +131,71 @@ non-prefix copy of a stream quarantines it loudly.
 
 ## Durability and concurrency
 
-Appends and merges run under a per-stream lockfile
-(`streams/<id>/.lock`, exclusive-create, stale after 30 s), because the
-torn-tail repair is a read-truncate-write sequence and hooks, watchers and
-manual passes can fire concurrently for the same owner. Sync tools may copy
-a `.lock` file; that cannot corrupt data, but a synced-in stale lock can
-delay a local `mergeStreamCopy` on that stream by up to the stale threshold
-(30 s), so excluding `.lock` from sync patterns is the right configuration. Journal writes fsync;
-note that on macOS `fsync(2)` does not guarantee media durability (that
-would need `F_FULLFSYNC`, which Node's fs API does not expose) — the
-guarantee is OS-crash-level, not power-loss-level.
+Appends and merges run under a per-stream lockfile (`streams/<id>/.lock`),
+because the torn-tail repair is a read-truncate-write sequence and hooks,
+watchers and manual passes can fire concurrently for the same owner.
+
+The lock carries an **owner token** — the holder's pid, its hostname, and a
+random nonce — written by hard-linking a fully-written temp file into place,
+so a lock that exists is always readable in full. Two guarantees rest on
+that token:
+
+- **A live holder is never stolen from, at any age.** Staleness is proven
+  from the holder's liveness, not from how long ago the lock was created.
+  A contender that finds the holder's pid alive on this host waits, and
+  fails with a timeout naming the holder — it does not reclaim. (The
+  earlier design judged a lock stale by age while the waiter's timeout
+  started when the contender arrived; a contender arriving late therefore
+  reclaimed a lock whose holder was still inside its critical section, and
+  both ran at once.)
+- **A holder releases only its own lock.** Release unlinks `.lock` only
+  while it still carries the releaser's nonce, so a holder whose lock was
+  reclaimed cannot delete the replacement lock out from under its new
+  owner.
+
+**Crash recovery is immediate, not timed**: a lock whose pid is gone from
+this host is provably stale and is reclaimed at once, without waiting out
+any threshold.
+
+A lock that cannot be read at all (anything but "not there" — a permission
+error, a directory in its place) is not retried: acquisition fails at once
+with the underlying error code and the lock's path, because retrying cannot
+clear such a condition. Everything else that fails to acquire — including a
+lock judged stale whose removal keeps failing — is bounded by
+`lockTimeoutMs` and reports why on timeout.
+
+`lockStaleMs` (30 s by default) is the fallback for the two cases where
+liveness cannot be checked here: a lock created on **another host** (file
+sync can copy one in) and a lock with **no readable token** (written by an
+older version of this package, or by hand). Those are still reclaimed by
+age. `lockTimeoutMs` (10 s) is how long a contender waits; the two
+thresholds are independent, and neither ordering of them is required or
+assumed.
+
+Honest limits:
+
+- **Pid reuse across a reboot.** A lock left by a process whose pid was
+  later reused by an unrelated live process reads as held forever, and
+  contenders on that stream time out until it is deleted by hand. The
+  timeout message names the lock path and the recorded holder. This is the
+  chosen direction of failure: refusing to write is recoverable, writing
+  concurrently with a live holder is not.
+- **Reclaim is not perfectly atomic.** A contender re-checks the lock's
+  identity (inode and mtime) immediately before removing it. That recheck
+  **detects** a replacement that landed while the old lock was being proven
+  stale — the common cascade, one contender reclaiming from whoever
+  reclaimed first — and the contender gives up and retries. It is a
+  detection, not an exclusion: a replacement landing in the window between
+  the recheck and the unlink is not seen and is removed. Nothing here
+  defends against that window.
+- **Sync tools may copy a `.lock` file.** That cannot corrupt data, but a
+  synced-in lock from another host is judged by age, so it can delay a local
+  `mergeStreamCopy` on that stream by up to 30 s — excluding `.lock` from
+  sync patterns remains the right configuration.
+
+Journal writes fsync; note that on macOS `fsync(2)` does not guarantee media
+durability (that would need `F_FULLFSYNC`, which Node's fs API does not
+expose) — the guarantee is OS-crash-level, not power-loss-level.
 
 ## Upgrading
 

package/packages/record/docs/turn-record-sot.md

@@ -8,7 +8,7 @@ aweb mail and chat messages into it. It is the contract implemented by the
 record tools (`capture`, `recall`, and the experimental tools over them); this
 package (`@awebai/turn-record`, in the oats repo) is the reference
 implementation, and the spec lives beside it on purpose: the record spans
-agent sessions from every harness, and the aweb server is one projected
+agent sessions from supported harnesses, and the aweb server is one projected
 source, not the record's home. The architecture decision it implements is
 `2026-08-18-turn-record-and-tools.md` in the strategy repo
 (github.com/awebai/strategy); the identity and messaging contracts it builds

package/packages/record/lib/store.mjs

@@ -19,6 +19,7 @@ import {
   existsSync,
   fsyncSync,
   ftruncateSync,
+  linkSync,
   mkdirSync,
   openSync,
   readdirSync,
@@ -28,8 +29,9 @@ import {
   statSync,
   unlinkSync,
   writeFileSync,
-  writeSync,
 } from "node:fs";
+import { randomBytes } from "node:crypto";
+import { hostname } from "node:os";
 import { join } from "node:path";
 
 import { finishTurn, sha256Hex, verifyTurnId } from "./canonical.mjs";
@@ -141,16 +143,186 @@ export function parseJournal(input) {
   return { turns, validEnd, torn: validEnd < bytes.length };
 }
 
+// ----------------------------------------------------------- stream lock
+//
+// The per-stream lock is an exclusive-create lockfile carrying an OWNER
+// TOKEN: the creating process's pid, its hostname, and a random nonce.
+// Two properties follow from the token that a bare lockfile cannot give:
+//
+//   - staleness is judged against the HOLDER'S LIVENESS, not the lock's
+//     age, so a contender that arrives late against a still-running holder
+//     waits (or times out) instead of stealing a live lock; and
+//   - release unlinks the lock only while it still carries the releaser's
+//     nonce, so a holder whose lock was reclaimed cannot delete the
+//     replacement lock out from under its new owner.
+//
+// Crash recovery is preserved: a lock whose pid is gone from this host is
+// provably stale and is reclaimed at once, without waiting out any age.
+//
+// Age remains the fallback for the two cases where liveness is not
+// knowable here: a lock created on another host (file sync can copy one
+// in) and a lock with no readable token (written by an older version, or
+// by hand).
+
+function newLockToken() {
+  return {
+    pid: process.pid,
+    host: hostname(),
+    nonce: randomBytes(12).toString("hex"),
+    acquiredAt: new Date().toISOString(),
+  };
+}
+
+// Create the lock, or report that someone else holds it. The payload is
+// written to a temp file and hard-linked into place, so the lock never
+// exists with partial or empty content: a contender that can see the file
+// can always read the whole token.
+function createLock(lockPath, dir, token) {
+  const tmp = join(dir, `.lock.tmp-${token.pid}-${token.nonce}`);
+  try {
+    writeFileSync(tmp, JSON.stringify(token) + "\n");
+    try {
+      linkSync(tmp, lockPath);
+      return true;
+    } catch (err) {
+      if (err.code === "EEXIST") return false;
+      throw err;
+    }
+  } finally {
+    try {
+      unlinkSync(tmp);
+    } catch {
+      /* never created, or already gone */
+    }
+  }
+}
+
+// Read the lock as { token, ino, mtimeMs }; token is null when the file
+// holds no readable owner token. Returns null when the lock has vanished.
+function readLock(lockPath) {
+  let st;
+  let raw;
+  try {
+    st = statSync(lockPath);
+    raw = readFileSync(lockPath, "utf8");
+  } catch (err) {
+    if (err.code === "ENOENT") return null; // vanished: the holder released
+    // Anything else (EACCES, EISDIR, EIO...) is a condition retrying cannot
+    // clear. Swallowing it here is what let the acquire loop spin: it read
+    // as "vanished" forever. Fail with the underlying code and path so the
+    // operator can act on it.
+    throw new StoreError(
+      `cannot read stream lock ${lockPath}: ${err.code ?? "unknown error"} (${err.message})`,
+      { cause: err },
+    );
+  }
+  let token = null;
+  try {
+    const parsed = JSON.parse(raw);
+    if (
+      parsed &&
+      typeof parsed.pid === "number" &&
+      typeof parsed.host === "string" &&
+      typeof parsed.nonce === "string"
+    ) {
+      token = parsed;
+    }
+  } catch {
+    /* no token: an older or hand-written lock, judged by age below */
+  }
+  return { token, ino: st.ino, mtimeMs: st.mtimeMs };
+}
+
+// true / false when the holder's liveness is knowable from this process,
+// null when it is not (the lock was created on another host). EPERM means
+// the process exists and belongs to another user: alive.
+function holderAlive(token) {
+  if (token.host !== hostname()) return null;
+  try {
+    process.kill(token.pid, 0);
+    return true;
+  } catch (err) {
+    return err.code === "ESRCH" ? false : true;
+  }
+}
+
+function lockIsProvenStale(held, staleMs) {
+  if (held.token) {
+    const alive = holderAlive(held.token);
+    if (alive === true) return false; // live holder: never stealable, at any age
+    if (alive === false) return true; // dead holder: crash recovery
+  }
+  return Date.now() - held.mtimeMs > staleMs;
+}
+
+// Remove a lock we have proven stale — and only that lock. Re-checking the
+// file's identity immediately before the unlink DETECTS a replacement that
+// landed while we were proving the old lock stale, so the common cascade
+// (reclaiming from whoever reclaimed first) is caught. It is a detection,
+// not an exclusion: a replacement landing between this recheck and the
+// unlink is still removed, and nothing here defends against that window.
+//
+// Returns null when there is nothing to report — the lock was removed, was
+// already gone, or had been replaced since we proved it stale; all three
+// mean "retry". Returns the underlying error when the removal itself keeps
+// failing (a read-only or hostile directory), which the caller reports on
+// timeout rather than retrying silently forever.
+function removeStaleLock(lockPath, held) {
+  let st;
+  try {
+    st = statSync(lockPath);
+  } catch (err) {
+    return err.code === "ENOENT" ? null : err; // already gone
+  }
+  if (st.ino !== held.ino || st.mtimeMs !== held.mtimeMs) return null; // replaced
+  try {
+    unlinkSync(lockPath);
+    return null;
+  } catch (err) {
+    return err.code === "ENOENT" ? null : err; // beaten to it
+  }
+}
+
+// Release: unlink only while the lock still carries our nonce. If it was
+// reclaimed (we ran long enough to be judged dead, or a sync overwrote
+// it), the file now belongs to another holder and is left alone.
+function releaseOwnedLock(lockPath, token) {
+  // Release runs in a finally block: it must not replace the critical
+  // section's own outcome with a lock error. A lock we cannot read is also
+  // a lock whose ownership we cannot prove, so leaving it is the correct
+  // move as well as the safe one — it is reclaimable by liveness once this
+  // process exits.
+  let held;
+  try {
+    held = readLock(lockPath);
+  } catch {
+    return;
+  }
+  if (held === null) return;
+  if (!held.token || held.token.nonce !== token.nonce) return;
+  try {
+    unlinkSync(lockPath);
+  } catch {
+    /* already gone */
+  }
+}
+
+function describeHolder(held) {
+  if (!held || !held.token) return "holder unknown";
+  return `held by pid ${held.token.pid} on ${held.token.host} since ${held.token.acquiredAt}`;
+}
+
 export class RecordStore {
   constructor(root, { owner, lockTimeoutMs = 10000, lockStaleMs = 30000 } = {}) {
     if (!root) throw new StoreError("record root is required");
-    // The lock has no liveness refresh: a holder that outlives lockStaleMs
-    // can be stolen from mid-critical-section. Keeping the waiter timeout
-    // strictly below the stale threshold guarantees a contender errors out
-    // loudly before it could ever steal a live holder's lock.
-    if (lockTimeoutMs >= lockStaleMs) {
-      throw new StoreError("lockTimeoutMs must be < lockStaleMs (stale-steal would race a live holder)");
-    }
+    // lockTimeoutMs is how long a contender waits; lockStaleMs is how old a
+    // lock must be before it is reclaimed WHEN ITS HOLDER'S LIVENESS CANNOT
+    // BE CHECKED (foreign host, no readable token). There is deliberately no
+    // ordering constraint between them: safety against stealing a live
+    // holder's lock comes from the owner token, not from the two thresholds
+    // being ordered — an ordering that never held, because the waiter's
+    // timeout starts when the contender arrives while age is measured from
+    // the lock's creation.
     this.root = root;
     this.owner = owner ?? null;
     this.lockTimeoutMs = lockTimeoutMs;
@@ -166,52 +338,44 @@ export class RecordStore {
   // processes: hooks, watchers and manual passes can all fire close
   // together for the same owner, and an unguarded repair can truncate a
   // concurrent writer's already-fsynced turn. Exclusive-create lockfile
-  // with stale-steal (a crashed holder's lock older than lockStaleMs is
-  // removed).
+  // carrying an owner token; see the stream-lock helpers above for how
+  // staleness is proven and why release is ownership-checked.
   withStreamLock(streamId, fn) {
     const dir = join(this.streamsDir(), streamId);
     mkdirSync(dir, { recursive: true });
     const lockPath = join(dir, ".lock");
+    const token = newLockToken();
     const deadline = Date.now() + this.lockTimeoutMs;
+    // Every iteration that fails to acquire falls through to ONE deadline
+    // check and one sleep — there is deliberately no `continue` above them,
+    // and no branch that retries "for free". An earlier version checked the
+    // deadline only on the readable-and-live branch, so a lock that kept
+    // failing to be read, or kept failing to be removed, or kept reading as
+    // vanished (a dangling symlink does exactly that) retried forever, hot,
+    // with nothing able to stop it. Retrying at once after a reclaim would
+    // buy ~25ms on the crash-recovery path and reopen that hole; it is not
+    // worth it.
     for (;;) {
-      try {
-        const fd = openSync(lockPath, "wx");
-        try {
-          writeSync(fd, String(process.pid));
-        } finally {
-          closeSync(fd);
-        }
-        break;
-      } catch (err) {
-        if (err.code !== "EEXIST") throw err;
-        let stale = false;
-        try {
-          stale = Date.now() - statSync(lockPath).mtimeMs > this.lockStaleMs;
-        } catch {
-          continue; // lock vanished between open and stat: retry now
-        }
-        if (stale) {
-          try {
-            unlinkSync(lockPath);
-          } catch {
-            /* another process stole it first */
-          }
-          continue;
-        }
-        if (Date.now() >= deadline) {
-          throw new StoreError(`timed out waiting for lock on stream ${streamId} (${lockPath})`);
-        }
-        sleepSync(25);
+      if (createLock(lockPath, dir, token)) break;
+      const held = readLock(lockPath); // throws on anything but ENOENT
+      const reclaimError =
+        held !== null && lockIsProvenStale(held, this.lockStaleMs)
+          ? removeStaleLock(lockPath, held)
+          : null;
+      if (Date.now() >= deadline) {
+        throw new StoreError(
+          `timed out waiting for lock on stream ${streamId} (${lockPath}, ${describeHolder(held)})` +
+            (reclaimError
+              ? `; it was judged stale but could not be removed: ${reclaimError.code ?? "unknown error"} (${reclaimError.message})`
+              : ""),
+        );
       }
+      sleepSync(25);
     }
     try {
       return fn();
     } finally {
-      try {
-        unlinkSync(lockPath);
-      } catch {
-        /* already stolen as stale; nothing to release */
-      }
+      releaseOwnedLock(lockPath, token);
     }
   }
 

package/capabilities/oats-aweb/package.json

@@ -1,20 +0,0 @@
-{
-  "name": "@awebai/oats-aweb",
-  "version": "0.2.0",
-  "description": "OATS messaging provider: per-instance aweb identities (mint at spawn, self-delete at retire), cross-machine roster, guided setup, and agents-md injection",
-  "keywords": [
-    "pi-package",
-    "oats",
-    "aweb"
-  ],
-  "repository": {
-    "type": "git",
-    "url": "https://github.com/awebai/oats",
-    "directory": "capabilities/oats-aweb"
-  },
-  "license": "MIT",
-  "type": "module",
-  "dependencies": {
-    "@awebai/pi": "^0.2.1"
-  }
-}

package/capabilities/oats-jira/package.json

@@ -1,25 +0,0 @@
-{
-  "name": "@awebai/oats-jira",
-  "version": "0.1.0",
-  "description": "OATS tasks provider: Jira via acli — epic/story/task protocol, agent roster, label-based agent identity, and agents-md injection",
-  "keywords": [
-    "pi-package",
-    "oats",
-    "jira"
-  ],
-  "repository": {
-    "type": "git",
-    "url": "https://github.com/awebai/oats",
-    "directory": "capabilities/oats-jira"
-  },
-  "license": "MIT",
-  "type": "module",
-  "bin": {
-    "oats-jira": "./bin/oats-jira.mjs"
-  },
-  "pi": {
-    "skills": [
-      "./skills"
-    ]
-  }
-}