@awebai/oats 0.41.1 → 0.42.0

package/docs/release-notes/v0.42.0.md

@@ -0,0 +1,398 @@
+# OATS 0.42.0
+
+## Added
+
+- **A capability can declare the home state it owns:
+  `retirement.disposable.home`**
+  ([#410](https://github.com/awebai/oats/issues/410)). The meaning is:
+  provider-owned state that is not the instance's work, left in place until
+  the home is removed, and not copied to recovery. The one exception is the
+  preservation of a failed spawn in directory mode, which copies the whole
+  home, declared entries included
+  ([#598](https://github.com/awebai/oats/issues/598)). A provider declares the
+  top-level entries of the instance home that hold its own state, such as an
+  identity directory with a signing key:
+
+  ```json
+  "retirement": { "disposable": { "home": [".aw", ".oats-aweb", ".aweb-identity", ".aweb-identity-*"] } }
+  ```
+
+  Each entry is one hidden top-level name (`.name`) or a prefix
+  (`.prefix-*`, every top-level entry whose name starts with the text before
+  `*`). Matching entries are left out of the home fingerprint, the home copy
+  and its verification, so a change to them alone causes no copy. Exclusion
+  means "not copied" and nothing more: the entries stay in the home until the
+  home is removed, retire hooks still see them, and an incomplete cleanup
+  keeps the home with them. The receipt names what was left out in
+  `workRecovery.notCopied` (`[{scope, path, owner}]`: names and owners only),
+  and the summary prints `not copied: .aw, .oats-aweb (oats.aweb)`. A
+  capability that declares nothing has its home state copied, once. This
+  kernel reads the field. See
+  [capabilities](../capabilities.md#manifest).
+
+## Changed
+
+- **Desktop has a theme picker (#602).** The theme button and
+  Ctrl+Shift+Space (⇧⌘Space on macOS) open a theme picker in the middle of
+  the window: the four themes, the current one marked. Choose one with a
+  click, Enter or Space; Escape closes it with nothing changed. The shortcut
+  works while a terminal has focus too. The theme button now opens the picker;
+  cycling between themes stays in the command palette (**Theme: cycle…**). On
+  Omarchy, Super+Ctrl+Shift+Space stays Omarchy's own theme menu (it changes
+  the computer's theme, which This computer follows); Ctrl+Shift+Space is its
+  in-app counterpart and opens Desktop's picker. A shortcut you had already set
+  to the same keys keeps them; the shortcuts editor names the clash.
+- **The Desktop accepts OATS CLIs `>=0.25.8 <0.43.0`**, so it runs against
+  this release's kernel; the Desktop 0.41.x refuses a 0.42 CLI.
+- **One retire writes one recovery**
+  ([#480](https://github.com/awebai/oats/issues/480),
+  [#544](https://github.com/awebai/oats/issues/544)). A retire whose hooks
+  changed the home or the work wrote a second recovery directory with a full
+  copy of both. It now adds to the recovery it already wrote, and copies again
+  only the part that changed:
+
+  ```text
+  <recovery>/
+    recovery.json
+    home/               the home before the retire hooks
+    repo/ or work/      the work before the retire hooks
+    after-hooks/
+      home/             the home again, only if a hook changed home bytes
+      repo/ or work/    the work again, unless it is proven unchanged
+  ```
+
+  The work is copied again unless it is proven unchanged: unchanged means
+  that everything a copy of it would carry is equal byte for byte, and
+  anything that cannot be compared exactly counts as changed. In worktree
+  mode that is what a work copy holds beside the files (status, branch and
+  commit; the index's entries, resolve-undo records and permission bits;
+  what it copies from the Git directories, with their permission bits: an
+  operation in progress, `info/attributes` and the stash's log; tags,
+  stash, exclude rules and status settings) and the bytes and permission
+  bits of its files. Every part is compared as its bytes, never as
+  decoded text, so two states that differ only in bytes that are not valid
+  UTF-8 (in a ref name, a path, a file name or the target of a symbolic
+  link) are two states. A recovery still cannot hold a file whose name is
+  not valid UTF-8: its copy fails and the retire refuses, as before.
+  A status with the same rows is not taken as proof, so a retire
+  hook that rewrites a file that was already modified has its bytes
+  preserved, whether or not it also writes the home
+  ([#600](https://github.com/awebai/oats/issues/600)). For that proof a
+  retire reads the worktree's files once before the hooks, when there is
+  something to preserve, and at most once more after them. A tag or a stash
+  made in the worktree's repository while the hooks run adds a copy
+  attempt. Two things are not compared, and the proof claims no byte
+  equality beyond them: the index's derived data (its stat cache and
+  extensions), a deliberate semantic exception; and the repository the
+  worktree belongs to (its objects, its other branches, the settings a clone
+  of it is served under), a custody boundary. It holds because of 0.41's
+  retirement ([#640](https://github.com/awebai/oats/issues/640)): the
+  repository survives the retire, no retire deletes a branch, and a commit
+  of the worktree that no ref reaches is preserved before the worktree is
+  removed.
+  A worktree that holds a repository (a directory under it with a `.git`
+  entry of any kind, a dangling symbolic link included; a repository made
+  by `git init` or a clone, a submodule, or a linked worktree of another
+  repository) is never proven unchanged: its work is copied again after
+  the hooks, whatever they did, and `afterHooks.work` then says that a copy
+  was made, not that a hook changed the work. That copy can fail where the
+  first did not, and the retire then refuses after the hooks with the home,
+  the work and the recovery written before them kept. A worktree with a
+  `.git` entry under it that OATS cannot read as a repository (a dangling
+  symbolic link, or a `.git` it cannot test) is never home-only, and no
+  class is added for it: a change to the home alone can now cause a
+  work-copy attempt before the hooks that OATS 0.41 did not make, and any
+  failure of that attempt can refuse the retirement, before any retire hook
+  runs and with the home and the work kept; a launched instance's session
+  has been stopped by then, as the refusal says. Of a nested repository's
+  branches other than the one `HEAD` is on, the copy holds the commits and
+  not the names: `git fsck --unreachable` in the copy lists them, and a
+  commit kept without a name is lost to `git gc` in the copy. Of its stash
+  the copy holds the latest entry; of its remote-tracking refs, notes and other
+  namespaces only what a branch or a tag reaches; a repository inside it
+  comes along as plain files.
+  A snapshot that holds the home only also gets the work under
+  `after-hooks/` when the hooks did not move it but something beyond the
+  home is there to preserve after them. The home's comparison holds every
+  entry its copy carries, the kernel's own records included
+  (`.oats-events.jsonl`, the `.oats-stop*.json` and `.oats-restart.json`
+  receipts, `.oats-agents-md.*.previous`, `.claude/settings.json`, and every
+  field of `instance.json`): a hook that writes one of them has the home
+  copied again. The home is not copied again because the work is, so when a
+  hook moved only the work those records are as of the pre-hook snapshot.
+  A home copy is still verified against the digest a spawn baseline uses,
+  which passes over those records: the verification does not prove they
+  were preserved ([#645](https://github.com/awebai/oats/issues/645)).
+
+  Each part under `after-hooks/` is whole and verified, verified
+  before the home is removed; `after-hooks/home/` exists only when the
+  home's own bytes moved. The pre-hook snapshot is never rewritten. An
+  instance that had nothing to preserve before the hooks still gets its one
+  recovery after them, with no `after-hooks/`. `recovery.json` gains `phase`:
+  `"before-hooks"` when the pre-hook snapshot is written, `"complete"` once
+  the post-hook check has concluded. A retried retire, after incomplete
+  cleanup or after a refused or interrupted attempt, writes its own recovery
+  and never touches an earlier one; a directory left at `before-hooks` is a
+  complete, verified pre-hook snapshot. The summary prints one "has been
+  preserved" block, with what was copied from the home and an `after the
+  retire hooks: … copied again under after-hooks/` line when it applies. See
+  [souls and instances](../souls-and-instances.md#retire).
+- **A socket, a FIFO or a device file in a worktree refuses the retire.**
+  In worktree mode, when a retire has something to preserve before its
+  hooks run, it now reads the whole worktree, and an entry that is not a
+  file, a directory or a symbolic link cannot be read or copied. Git prints
+  no status row for such an entry. Before, a retire refused it only inside
+  a directory Git reports as ignored, or when it copied the work; a retire
+  that preserved only the home went through. Now it refuses with `E_WORK_INSPECTION_FAILED` before
+  any retire hook runs: no recovery was written and nothing was deleted,
+  however often it is retried. The message names the entry and says what to
+  do: safely stop the process or resource that owns it, or move the entry
+  elsewhere, then retry (it may be a live endpoint, so the advice is not to
+  delete it).
+  `--json` keeps the code; the `message` field is the new sentence, also for
+  the cases that were already refused. Those are refused where they were,
+  at the first inspection: an entry inside a directory Git reports as
+  ignored whole, in the `work/` of a directory instance, or in the home
+  itself. That inspection runs before the retire stops the session, so
+  there the session is still running, and the message ends after saying
+  what to do. Everywhere else in a worktree the refusal comes from the new
+  read, after the retire has stopped the session of a launched instance:
+  the instance's session is stopped, the instance is not retired, and its
+  home is kept; the message says so, and says that this retire stopped no
+  session when the instance had none to stop. To continue, deal with the
+  entry and run
+  `oats retire <instance>` again, or start the session again in the same
+  home with `oats session start --home <abs>`. A self-retire (`--self`) is
+  refused the same way by its detached completion, which records the
+  refusal beside the home; after that only `oats retire <instance>`
+  continues, and `oats session start` refuses until it has. Of the refusals
+  from the new read, only that of `--self --keep-dir` comes with the session
+  still running. When a retire hook leaves such an
+  entry behind, the hooks have run: the retire refuses
+  (`E_WORK_INSPECTION_FAILED`, or `E_WORK_PRESERVATION_FAILED` when the hook
+  also moved the Git state) and the home, the work and the recovery written
+  before the hooks are kept.
+- **A file in a worktree that cannot be read refuses a retire that
+  preserved only the home.** In worktree mode, when a retire has something
+  to preserve before its hooks run, it reads every file of the worktree,
+  also what is nothing to preserve: a tracked file that is unchanged, and a
+  file under a work root a capability declared disposable
+  (`retirement.disposable.work`). A file without read
+  permission, or one over 2 GiB (which a single read cannot take), refuses
+  the retire with `E_WORK_INSPECTION_FAILED` before any retire hook runs,
+  with or without `--force`: no recovery was written and nothing was
+  deleted. Before, a retire whose snapshot held the home only did not read
+  the work and went through. The message is `could not read the worktree at
+  <work>: <reason>`: for a file without read permission the reason names the
+  file; for a file over 2 GiB it gives the size, and
+  `find <work> -type f -size +2147483647c` finds the file. To continue, move
+  the file out of the worktree or make it readable, then run
+  `oats retire <instance>` again. As for a socket or a FIFO that the new
+  read meets, the session of a launched instance has already been stopped by
+  then, the instance is not
+  retired, and its home is kept.
+- **A worktree that cannot be proven unchanged is copied before the hooks,
+  and a retire of one that preserved only the home can now be refused.**
+  A worktree is not provable when it holds a repository (a `.git` entry of
+  any kind under it, a dangling symbolic link included), or when a read of
+  its Git state fails while `git status` works (for example
+  `info/attributes` or the stash's log cannot be read). Such a worktree
+  always has its work in the pre-hook snapshot and is copied again after
+  the hooks; with nothing to preserve it retires as before. The newly
+  refused case: a retire whose pre-hook snapshot would have held the home
+  only, whose worktree has a state read that fails or a `.git` entry under
+  it that OATS cannot read as a repository, and whose copy cannot be made
+  either. It refuses with `E_WORK_PRESERVATION_FAILED` before any
+  retire hook runs: no recovery was written and nothing was deleted; the
+  session of a launched instance has already been stopped, the instance is
+  not retired, and its home and work are kept. Before, that retire went
+  through and the work was never read. A retire hook can leave the worktree
+  in that state too: the work is then copied under `after-hooks/`, also when
+  the pre-hook snapshot held the home only, and when that copy cannot be
+  made the retire refuses with `E_WORK_PRESERVATION_FAILED` after the hooks
+  have run, with the home, the work and the recovery written before the
+  hooks kept. Before, that retire went through as well.
+- **A retire refused by the copy made before its hooks says how far it
+  got.** The message of every `E_WORK_PRESERVATION_FAILED` raised by that
+  copy, whatever its cause (a recovery that could not be verified, an entry
+  that cannot be copied), now ends with: no retire hook has run, no recovery
+  was written and nothing was deleted, the instance is not retired, its home
+  and work are kept, and either "its session has been stopped" or "this
+  retire stopped no session". `--json` keeps the code and `details`; the
+  `message` field is longer. A refusal after the hooks is unchanged.
+- **A retire that already refused an entry it cannot read now refuses it
+  with another error code.** A worktree that the retire copies before its
+  hooks (it has work to preserve) and that holds an entry that is not a
+  file, a directory or a symbolic link was refused with
+  `E_WORK_PRESERVATION_FAILED`, when the copy met the entry. It is now
+  refused earlier, when the worktree is read, with
+  `E_WORK_INSPECTION_FAILED`; the message names the entry's path and says
+  what to do. A worktree that cannot be proven unchanged is copied without
+  that read, and keeps `E_WORK_PRESERVATION_FAILED`. The home is kept in
+  both.
+- **`oats retire --json` no longer emits `workRecoveries`.** The receipt has
+  one `workRecovery`, which keeps `path`, `classes`, `bytes`, `outputs` and
+  `repoCopy` and gains `home` (what was copied from the home), `notCopied`
+  and `afterHooks: {home, work}` (which parts are under `after-hooks/`).
+  A reader that consumes the receipt should read `workRecovery`, and keep
+  accepting `workRecoveries[]` from an older kernel on a server. The
+  Desktop's reader of the receipt is unchanged. See
+  [the CLI API](../desktop-cli-api.md#retire).
+- **`oats retire <instance> --plan` says what a retire would copy.** Two
+  `notes` are added: where changed home files are copied, with the declared
+  entries that are not copied, and whether the work is copied too (worktree
+  and directory modes). No other key changes, and `planRevision` is
+  unaffected.
+- **A forced removal past incomplete cleanup no longer leaves a copy of
+  declared state in recovery.** `oats retire <instance> --force` on a home
+  whose cleanup cannot finish removes the home after preserving its work.
+  Entries declared in `retirement.disposable.home` go with the home; the
+  recovery holds no copy of them.
+- **A malformed `retirement` is refused where the manifest is read.** Member
+  discovery (`E_WORKSPACE_SCHEMA`), package manifests (`E_PACKAGE_MANIFEST`)
+  and the kernel loader refuse the same manifest with the same JSON pointer.
+  `retirement` must contain a `disposable` map with only `home` and `work`,
+  each an array of strings. A `home` entry must be one hidden top-level name
+  (`^\.[A-Za-z0-9_][A-Za-z0-9._-]*$`) or a prefix
+  (`^\.[A-Za-z0-9_][A-Za-z0-9._-]*-\*$`). Refused: anything else (`notes`,
+  `.aw/keys`, `.a*`); an exact name that is `.oats`, `.agents` or `.claude`,
+  or that starts with `.oats-events`, `.oats-stop`, `.oats-restart`,
+  `.oats-rollback`, `.oats-agents-md`, `.oats-start` or `.oats-attachments`
+  (the entries the kernel itself writes in a home); and a prefix that starts
+  with `.oats-`. An exact name such as `.oats-aweb` is allowed. The published
+  [manifest schema](../capability-manifest.schema.json) states the same
+  rules.
+
+  **This is stricter than before for `home`.** Before this release `home`
+  entries were not checked against a grammar; an entry outside the grammar
+  now refuses the whole manifest. For a home whose module copy
+  (`<home>/.oats/modules/<capability>/oats.json`) carries such an entry, the
+  commands that load the home's module manifests stop: starting or
+  restarting its session, and capability commands run from the home.
+  `oats inspect` reports it as a `module-manifest-invalid` problem instead of
+  failing. Retiring such a home still works, because retire runs the hooks
+  recorded in `instance.json` at spawn and does not load the manifests. No
+  capability shipped in this repository declares `retirement`.
+
+- **A pane's environment is its tmux session's, `PATH` included, whoever
+  opens its window** ([#616](https://github.com/awebai/oats/issues/616)).
+  The tmux client that creates an agent's window, or respawns its pane in a
+  restart in place, now runs with only `LANG`, `LC_ALL` and `LC_CTYPE`, for
+  every creator: an instance, the Desktop, an operator's shell. tmux then gives
+  the pane its session's `PATH`, or the server's global one, instead of the
+  creating process's. Before, a pane's `PATH` was the `PATH` of each `oats
+  spawn` or `oats session start`, while the rest of its environment was the
+  server's: a version-manager wrapper (an Omarchy `~/.local/bin/claude`,
+  which runs `mise x claude -- claude`) then saw a `PATH` without the
+  variables that interpret it, resolved its own name again and looped. A
+  `PATH` and its companion state now come from one environment. OATS does not
+  certify that environment: an existing or external server, a session's own
+  `PATH` and a launch configuration's `PATH` are used as they are. The tmux
+  client is run by the absolute `tmux` the creating process's own `PATH`
+  finds. See [execution targets](../execution-targets.md#the-servers-start-environment).
+- **The harness is looked up on the `PATH` its pane runs it with**
+  ([#616](https://github.com/awebai/oats/issues/616)). A spawn or a start
+  that looks its executable up by name (the harness's, or a launch
+  configuration's bare `executable`) uses, in this order: a declared absolute
+  `executable` as it is; the launch configuration's own `PATH`; otherwise the
+  session's or the server's `PATH`, never the creating process's. It looks
+  once before anything is created, against the `PATH` it expects, and again
+  on the session it actually gets (another creator may have started the
+  server, a session may override `PATH`): found elsewhere, that executable is
+  launched and recorded; not found, the spawn is rolled back. With no server
+  running, a start under a new selection looks it up on the `PATH` of the
+  environment the server will be created with. A start that reuses the
+  home's recorded pane looks on that pane's own session, wherever it is
+  recorded, never on the OATS server. A session that
+  clears `PATH`, a `PATH` the strict reader cannot read and a server with no
+  `PATH` are refused, never substituted. The refusal is
+  `E_HARNESS_UNAVAILABLE` (`E_LAUNCH_EXECUTABLE` for a configuration's
+  declared name); its message names where it looked and the remedies
+  (declare `executable`, set `PATH` in the launch configuration, or start the
+  server from your own shell) and prints no `PATH`. A recorded executable,
+  reused by a plain start, is not looked up again.
+- **When OATS starts the `oats` tmux server, it gives it your login
+  environment** ([#616](https://github.com/awebai/oats/issues/616)). Whoever
+  runs the command that starts it (an instance, the Desktop, an operator's
+  shell, a schedule runner), OATS runs your login shell from the password
+  database (bash, zsh or fish) as `-l -i -c` from a fixed seed (your `HOME`,
+  `USER`, `LOGNAME` and `SHELL`, a minimal `PATH`, `TERM=dumb`, the creator's
+  locale, and the session variables `SSH_AUTH_SOCK`, `DISPLAY`,
+  `WAYLAND_DISPLAY`, `XDG_RUNTIME_DIR` and `DBUS_SESSION_BUS_ADDRESS` from
+  the creator when it is no instance, the server an instance's home records,
+  or your user session), and starts the server with the environment that
+  shell sets up. Nothing else of the creator, so no agent identity,
+  credential or harness variable, reaches the server, except your OATS
+  configuration: the `OATS_` and `PI_AGENTS_` variables that are not the
+  kernel's (`OATS_HOME_DIR`, `OATS_TMUX_SESSION`, …) stay the creator's (an
+  instance's: its recorded server's), as they were before. The functions
+  bash exports (`BASH_FUNC_<name>%%`) are dropped. The login shell is run
+  only for a process whose `HOME` is your home directory; one that set
+  another `HOME` takes the fallback. The answer comes as one frame on the
+  shell's stdout, marked with a random value made for each reading (never a
+  file, and not a descriptor of its own, which bash 5.3 started `-l -i` closes
+  before the emitter runs); everything else the shell prints is discarded,
+  and a missing, cut or repeated frame is no answer; it is accepted only
+  whole (exit 0, at most 1 MiB, one JSON object of text, `HOME` and `PATH`
+  present) and never evaluated; the read is bounded at 5 s and the
+  shell's process group is killed at the deadline. Before, the server got the
+  creating process's environment (an instance: a copy of its recorded
+  server's). See [execution targets](../execution-targets.md#the-servers-start-environment).
+
+## Fixed
+
+- **No classic scrollbars in the Desktop on Linux.** Chromium on Linux drew a
+  classic scrollbar, arrows included, under the terminal tabs and beside a
+  terminal. The tab strips still scroll (the wheel, and to show the selected
+  tab) without a scrollbar, like the other tab bars, and a terminal's pane no
+  longer scrolls: the terminal scrolls itself. macOS showed neither.
+- **A copy keeps the target of a symbolic link as its bytes.** A recovery,
+  and every other tree the kernel copies (a skill at spawn), recreates a
+  symbolic link whose target is not valid UTF-8 with the target's bytes.
+  Before, each such byte became a replacement character, and the link in the
+  copy named another path. A target that is valid UTF-8 is recreated as
+  before.
+
+## Notes
+
+- **No manual step for the `oats` server any more.** 0.41.0 asked you to
+  start it from your own shell before anything else did. Now, when OATS
+  starts it (after a reboot, or when its last session ends), it reads your
+  login environment itself. When that cannot be read, OATS says so in one
+  line on stderr, with no value, and falls back: a command run outside every
+  instance uses its own environment, as before; an instance uses a copy of
+  the server its home records, or is refused. A fallback is degraded: it is
+  not a login environment and does not prove a working agent socket, and it
+  passes the creator's environment on as it is, so a creator whose `PATH`
+  names a version manager's directories without the variables that interpret
+  them can still make a wrapper such as Omarchy's loop. Only a login
+  environment that OATS read prevents that.
+- **A server you or a service started is used as it is.** A server you start
+  yourself (`tmux -L oats new-session …`) keeps your shell's environment; one
+  a service manager starts (`systemd-run`, a unit, launchd) gets only that
+  manager's environment, and OATS does not change or certify it. A server
+  that is already running is not restarted: the login environment applies the
+  next time OATS starts the server.
+- **The pane rule applies to every window created after the upgrade, on
+  whatever server is running, for every creator: instances, the Desktop and
+  an operator's shell.** No pane is killed or migrated, and a running agent
+  keeps its environment. This is a behaviour change: a new pane's `PATH` is
+  now the server's (or its session's), no longer the creating process's, and
+  a harness found only on the creating process's `PATH` is refused with
+  `E_HARNESS_UNAVAILABLE`. On a server a service manager started
+  (`systemd-run`, a unit, launchd), whose `PATH` is only that manager's, a
+  spawn or start by bare harness name, an instance's included, is refused
+  until you act. To give the panes another `PATH`, set it on the server
+  (`tmux -L oats set-environment -g PATH "$PATH"`: windows created from then
+  on take it), declare the executable or a `PATH` in the launch
+  configuration, or let OATS start the server (it then gets your login
+  environment) or start it from your own shell.
+
+- **Instances spawned before the upgrade retire with one copy, and still
+  copy provider state.** The declaration is recorded at spawn in the home's
+  retirement baseline, and retire reads it from there only. A home spawned
+  before its capability declared the entries gains no exclusion from a
+  package update: its home is copied whole, once. An instance spawned after
+  the capability declares them leaves them out.
+- Existing recovery directories are not read, listed or deleted by this
+  release.