agentus 0.0.0-stage → 0.2.0

package/docs/dev/04-recover.md

@@ -0,0 +1,199 @@
+# Recovering when it breaks
+
+This doc answers: the instance is broken or misbehaving — where do I look first, and how do
+I fix it without making the box worse?
+
+## Read the boot log first
+
+The boot log is the authoritative account of what the server resolved. It prints **which
+data directory won and why**, the auth state, and the TLS listener:
+
+```
+data dir: /Users/…/packages/server/.data  (AGENTUS_DATA was set)      ← the dir + the reason
+auth on: user "admin"
+https://0.0.0.0:8443 (self-signed …)                                  ← present = TLS listener up
+tls off: no cert at <DATA_DIR>/tls/cert.pem                           ← absent/off = no TLS
+```
+
+If the directory it names is not the one you expected, stop here — a server serving the
+wrong store looks healthy (200s, valid `/api/version`) while the operator's session list is
+empty. Triage the identity, not the liveness (see "Is it serving the wrong store?" below).
+
+## Ports
+
+| port | who |
+|---|---|
+| `8787` | live HTTP |
+| `8443` | live TLS (public tunnel points here) |
+| `8901` | dev / QA instance |
+| `8648` | **Hermes Studio** — not Agentus, do not touch |
+
+```bash
+lsof -nP -iTCP:8787 -sTCP:LISTEN        # who owns the port (returns the CHILD, not the supervisor)
+ps -o pid,ppid,pgid,command -p <pid>    # its process group and command line
+```
+
+## "Address already in use"
+
+`EADDRINUSE` means another process holds the port — often a previous instance that never
+died, or (this box) an ambient `PORT=8648` pointed at Studio. The Agentus server reads
+`AGENTUS_PORT`, never bare `PORT`, so a script that reads the ambient value will fight for
+Studio's port. Do not just relaunch: identify the current holder, decide whether it is the
+one you want, then stop it through its own launcher.
+
+```bash
+# clean stop of the dev instance (kills the process GROUP on :8901)
+bash scripts/dev.sh stop
+```
+
+For live, stop it only via the release path ([03-release.md](./03-release.md)) — a bare kill
+of the live listener ends the operator's sessions.
+
+## Orphaned agent and shell child processes
+
+Every agent child is spawned detached and its pid recorded, and the server reaps orphans on
+next boot — so a crashed server's leftovers are cleaned up automatically. Shell sessions
+(the workspace terminal) are killed with their socket. Problems appear when a **supervisor**
+(the `tsx watch` wrapper) survives a crash: the listener it spawned may be gone while the
+supervisor sits in a restart loop, or a leaked listener keeps holding the port.
+
+**Never kill by a substring match.** A cleanup filtered on `packages/server/src/index.ts`
+once ended 249 processes in one shot — the operator's own `hermes-studio` checkout has a
+file at that same path, and its server was the only way into the machine. Rules:
+
+1. **Enumerate first and print the full command line and cwd** of every candidate:
+   ```bash
+   ps -eo pid,ppid,pgid,command | grep <pattern>
+   lsof -a -p <pid> -d cwd -Fn      # where it is actually running
+   ```
+2. **Keep the list inside this worktree's path.** The operator's services are never in
+   scope. A `ps | grep` list also contains your own tool shell — exclude the current pid and
+   its ancestors, or an unfiltered kill ends your own command mid-flight.
+3. **Prove the target is its own process group, then kill the group:**
+   ```bash
+   ps -o pid,pgid,sess -p <pid>
+   kill -9 -- -<pgid>              # negative pid = the group
+   ```
+   Under a supervisor the LISTEN pid is the supervisor's child, whose pgid is the
+   supervisor's — killing only the child leaves the supervisor (which respawns and fights
+   for the port). This is why `dev.sh`/`relaunch.sh` kill `-$pgid` and, when `pgid != pid`,
+   the pid too.
+4. **Fix the leak, not the puddle.** New scripts must use the shipped pattern —
+   `spawn(..., { detached: true })` and kill the process group — rather than cleaning up by
+   name afterwards.
+5. **Re-check the neighbours afterwards:** one `curl` per resident service
+   (`:8648` Studio, `:8787`/`:8443` live, `:8901` dev). A cleanup that leaves the box worse
+   than it found it is the whole failure mode.
+
+## The SQLite store, WAL, and the `*.bak` files
+
+State lives in `<DATA_DIR>/agentus.sqlite` (the transcript and every other row), accessed
+through Node's built-in `node:sqlite` (hence Node `>= 22.5`). Files beside it: `credentials.json`,
+`auth.token`, `auth.secret`, `sessions.json`, `settings.json`, `tls/` — several of them
+`0600`. The server creates the directory `0700` on first boot.
+
+**Back up before you touch the store.** A single write against the operator's real data gets
+a backup and a record of the old values:
+
+```bash
+cp -p packages/server/.data/agentus.sqlite \
+      ~/.hermes/cache/scratch/agentus.sqlite.bak-$(date +%H%M%S)
+```
+
+The one migration path in the project that rewrites rows in place — the legacy transcript
+fold — takes its own backup first with `VACUUM INTO <db>.pre-fold.bak` and gates itself on
+`pragma user_version`, so it runs once and is idempotent. If you see a `*.pre-fold.bak`
+beside the store, that fold ran; the original is inside it.
+
+Two cautions that come from real incidents:
+
+- **Two independent writers on one WAL database** corrupted the operator's live `state.db`
+  while the runtime linked a SQLite in the WAL-reset bug range. The fix is the interpreter
+  (link SQLite ≥ 3.51.3), not a symlink, and it belongs to the Hermes Studio runtime, not to
+  this repo. Check it with `hermes doctor` and read the ⚠ lines — a warning is real, not
+  imaginary, but the shipped `hermes doctor` is warn-only. Never add a second writer to the
+  live home on a vulnerable build.
+- `DELETE /api/sessions/<id>` **closes a live session and purges a cold one** — for an
+  already-dead row it deletes the record. So "archive this dead session" is not that route;
+  it takes a direct, backed-up, single-row transaction (`WHERE id=? AND status='error'`), no
+  live row touched, and no restart needed (a cold row is not in the server's in-memory map).
+
+## Is it serving the wrong store?
+
+Symptoms that look like separate bugs but share one cause: the operator's session list came
+back empty, TLS silently turned off (`tls off: no cert at <QA_DIR>/tls/cert.pem`), and a
+running slot was reaped — yet the port answers 200 and `/api/version` looks fine. Cause:
+`relaunch → launch.py` inherited the QA environment. The launcher now pins its own
+`AGENTUS_DATA`/`AGENTUS_PORT` and pops the QA/DEV keys, but verify identity directly:
+
+```bash
+PID=$(lsof -nP -iTCP:8787 -sTCP:LISTEN -t)
+ps eww -p "$PID" | tr ' ' '\n' | grep AGENTUS          # the process's own identity vars
+lsof -p "$PID" | grep sqlite                           # which store it actually opened
+```
+
+Recovery:
+
+```bash
+cd worktrees/agentus
+env -u AGENTUS_DATA -u AGENTUS_BASE -u DEV_LOG_PATH -u DEV_PROMOTE_DETACHED -u DEV_PROMOTE_SELF \
+  bash scripts/relaunch.sh --port 8787 --launcher ~/.hermes/cache/agentus/launch.py
+TOK=$(cat packages/server/.data/auth.token)
+curl -s -X POST -H "Authorization: Bearer $TOK" http://127.0.0.1:8787/api/sessions/<id>/resume
+```
+
+After any restart, put the operator back on the air: `POST /api/sessions/<id>/resume` with the
+machine token → `status: ready` and a fresh pid. **Capture the slot ids to resume before the
+restart** — in the window right after a restart a slot has dropped from `live` into
+`archived` wearing `error`, so a resume loop that reads the current `live` list finds
+nothing. `resume` works on an `archived('error')` row; that is the cold-slot entry point.
+
+## Auth lockout
+
+The login limiter is visible and reversible without a restart, so locking yourself out is a
+one-click problem. Endpoints behind the login:
+
+```bash
+TOK=$(cat <DATA_DIR>/auth.token)
+curl -s -H "Authorization: Bearer $TOK" http://127.0.0.1:8787/api/auth/locked-ips
+curl -s -X DELETE -H "Authorization: Bearer $TOK" http://127.0.0.1:8787/api/auth/locked-ips
+```
+
+In the UI it is Settings → 登录失败锁定 (which IPs are locked, for how long, unlock / unlock
+all). Tunables: `AGENTUS_LOGIN_MAX_FAILS` (default 5) / `AGENTUS_LOGIN_LOCK_MS` (default
+30000). Login sessions are stateless signed cookies indexed at `<DATA_DIR>/sessions.json`
+(0600) — advisory for display, authoritative for existence. A password change invalidates
+every other session; `AGENTUS_AUTH=off` disables the login entirely and the boot log scolds
+you (do not leave it off — the listener is `0.0.0.0` and this cockpit is tunnelled).
+
+## TLS certificate regeneration
+
+The TLS listener runs on a **second port** (8443 live), pointed at by a plain-TCP tunnel.
+Cert material lives at `<DATA_DIR>/tls/`. Regenerate with:
+
+```bash
+scripts/make-cert.sh [name]        # default name: the DDNS name; SAN = that name + localhost + 127.0.0.1
+scripts/make-cert.sh --leaf-only   # rotate the leaf only; installed devices are untouched
+```
+
+`make-cert.sh` issues **two** certs on purpose: a 10-year root (`ca.pem`, what devices install
+once, served at `/cert.crt`) and a 390-day leaf (`cert.pem`/`key.pem`, what the listener
+serves, under Apple's 398-day cap). Keep the SAN to the DDNS name you actually type, not the
+dynamic public IP. It writes into the same directory the server reads from (mirroring the
+server's own resolution in `packages/server/src/index.ts`), and if that resolution and the
+server's ever diverge the listener stays silently off with a cert sitting next to it — after
+regenerating, confirm the boot log still prints `https://…:8443`.
+
+## The process-exit safety net for children
+
+The server installs `SIGINT`/`SIGTERM`/`SIGHUP` handlers that terminate the child agent
+processes, so Ctrl-C (or a deliberate restart) takes its agents with it. The
+`unhandledRejection`/`uncaughtException` guards are installed **only after `listen()`
+succeeds** — installing them earlier would turn a startup failure (`EADDRINUSE`) into a
+zombie that is "alive but not listening". Combined with the on-boot orphan reap, this is
+what keeps a crashed server from leaving runaway children behind. If you find runaway
+children anyway, hunt them by the process-group rule above — never by name.
+
+Cross-links: [01-dev-loop.md](./01-dev-loop.md) (ports, data dirs) ·
+[03-release.md](./03-release.md) (release + rollback) ·
+[02-verify.md](./02-verify.md) (stray CDP tabs, leaked servers).

package/docs/dev/05-agent-driven.md

@@ -0,0 +1,114 @@
+# Developing Agentus with a coding agent in the loop
+
+This doc answers: how do I hand a task on this repository to a coding agent, and how do I
+review what it produces?
+
+## AGENTS.md — the file an agent reads first
+
+`AGENTS.md` is a plain-markdown conventions file that coding agents (Hermes, Qoder CLI and
+others) read automatically at the start of a session. It is where a repo states the things
+that are true of every change: where things live, the invariants, the git discipline, the
+commands to run. It is not documentation for humans — it is the brief an agent gets before
+it touches anything.
+
+**The Agentus repository itself carries no `AGENTS.md` as of this writing** — the
+conventions that govern work on it live in this `docs/dev/` set plus the operator's own
+working notes. `TODO(verify)`: a `docs/dev/`-aware root `AGENTS.md` may be added; if one
+appears it becomes the entry point an agent reads first, and this manual is what it links to.
+
+Where `AGENTS.md` **is** load-bearing in this workspace: the operator's spoken-reply
+workspace (`~/Workspace/voice-chat/AGENTS.md`) is read by both `hermes` and `qodercli`, so
+the same style rules apply whichever backend a session runs on. A running session keeps the
+prompt it booted with, so an edit to that file only shows up in a **new** session. When a
+task is about how Agentus-backed sessions should *speak*, that file — not the code — is the
+lever.
+
+## Handing a task to an agent
+
+Agentus is a good target for agent work because every claim is checkable in-repo. Give the
+agent four things and it can work without guessing.
+
+**1. The real entry points.** Name the files, do not describe them:
+
+| area | file |
+|---|---|
+| server entry | `packages/server/src/index.ts` (boots, resolves data dir, wires routes/WS) |
+| spawn + ACP relay | `packages/server/src/acp/registry.ts`, `acp/backends.ts` |
+| store | `packages/server/src/store/store.ts` |
+| auth | `packages/server/src/auth.ts` |
+| web app | `packages/web/src/App.tsx`, `state.ts`, `transcript.ts` |
+| mock agent (dev/QA) | `packages/server/mock/agent.mjs` |
+| dev launcher | `scripts/dev.sh` · release: `scripts/relaunch.sh` · `scripts/make-cert.sh` |
+
+**2. The reference implementation to compare against.** Almost every UI question has a
+sibling project that already answered it, and the operator expects behaviour parity, not
+copied markup. Point the agent at the right checkout (below) and ask for the *contract* —
+where the control lives, what it reads, how it behaves — never the markup.
+
+**3. The verification the change must run.** State it up front so the agent does not declare
+victory on a build:
+
+- `npm run typecheck` (0 errors), then `npm run build`;
+- the server smokes for the area (`node scripts/*.mjs`);
+- for a UI change, a CDP sweep **and** `npm run build` before it (the server serves the built
+  bundle). If no sweep covers the surface, the round writes one — see [02-verify.md](./02-verify.md);
+- for a release, the deployed-bundle checks in [03-release.md](./03-release.md).
+
+**4. The boundary.** The live instance may be restarted only by an explicit release, and
+only on the operator's word unless the change is strictly web-only. Say this in the brief;
+an unprompted `promote` kills the session the agent is running in.
+
+## Reference implementations, and where they live
+
+The operator asks to align a surface with these two products ("体验对齐" = behaviour parity).
+Read the checkouts before designing; take the contract, not the code — a genuinely derived
+block needs its license/NOTICE line, an idea needs nothing.
+
+| product | checkout | what it is good for |
+|---|---|---|
+| AionUi | `~/Project/AionUi` (`packages/desktop/src/renderer`) | layout mechanics: resizable panes, grouped history, local-file preview, table zoom, markdown utils; its `docs/prds/**` give per-surface intent *and* a "已知局限" appendix (the admitted flaws are what NOT to copy) |
+| hermes-studio | `~/.hermes/cache/studio-ref` (client) | the session menu, tool-run grouping (`ToolRunSummary.vue`, `tool-run-grouping.ts`), the message queue, `appendedTextDelta` streaming |
+
+Repo-side notes on both are under [`docs/refs/`](../refs/) (`aionui-acp-rules.md`,
+`hermes-studio-ui-ideas.md`).
+
+Two rules that come out of using them:
+
+- **Code, not screenshots.** A reference's source gives the decisions (e.g. "running tools
+  are not grouped"; the exact `useResizableSplit` clamping) that a screenshot hides.
+- **Check the form before borrowing.** AionUi is an Electron app; a mechanism that hangs off
+  its Titlebar has no equivalent in a pure web page + Android WebView shell. Ask "does it
+  attach to a layer we have?" before copying it.
+
+## Review discipline
+
+These are the habits that keep agent-produced changes safe on a two-instance, live-traffic
+codebase.
+
+- **Diagnose before editing.** For a reported defect or a "why did it do that" question,
+  produce the mechanism (store query, log line, code path) and the numbers **first**, and
+  stop there. A diff in the same turn as a question is answering a question with
+  unrequested work. When a spec is restated, write your reading back — the shape, one
+  example, and every point where your reading could differ — before touching code.
+- **Root cause over symptom, and fix the class.** The operator reports the symptom
+  accurately and the cause never. Find the general condition, not just the instance the
+  report names, and say which class of defect it belongs to.
+- **One logical change per commit.** Write the message to a file and use `-F` (inline
+  backticks / `$` / `!` are eaten by the shell). If a file carries someone else's in-flight
+  work, commit only your hunk (`git diff -U3 -- <file>` → `git apply --cached <patch>`),
+  never a plain `git add <file>`.
+- **Reproduce before you fix; make the new check prove the bug.** Run the assertion against
+  the current (broken) build, quote the failing count, then fix, then re-run. A check that
+  never failed on the broken build proves nothing.
+- **Measure before and after, and put the numbers in the reply.** "Looks the same to me" is
+  not evidence, and a light/dark screenshot pair can be two copies of one theme — compare
+  real values.
+- **Re-verify the checks that watched the thing you changed.** A behaviour change
+  invalidates its own observation point; a harness edit invalidates every green it produced.
+- **Leave the cockpit as you found it.** Purge the sessions, workspaces and probe files the
+  round created; keep the operator's; report the final counts.
+- **`main` and the live instance move only on their own paths** — merge/release per
+  [03-release.md](./03-release.md), never as a side effect of a "fix".
+
+Cross-links: [01-dev-loop.md](./01-dev-loop.md) · [02-verify.md](./02-verify.md) ·
+[03-release.md](./03-release.md) · [04-recover.md](./04-recover.md).

package/docs/dev/README.md

@@ -0,0 +1,65 @@
+# Developing Agentus
+
+This is the in-repo manual for working on Agentus: how to run a development instance,
+how to prove a change works, how to release it to the live instance, and how to recover
+when it breaks.
+
+**What Agentus is.** A multi-session web cockpit for ACP-speaking coding agents
+(`hermes acp` / `qodercli --acp`). The server spawns one child process per session and
+relays ACP JSON-RPC between the browser and that agent. The UI holds no intelligence:
+gate access → spawn → relay → stream → persist → surface permission cards. State lives in
+`<DATA_DIR>` (default `~/.agentus`; see [01-dev-loop.md](./01-dev-loop.md) for how the
+directory resolves). Node `>= 22.5` is required — the store is `node:sqlite`.
+
+## The mental model
+
+There are **two instances of the same codebase**, distinguished only by environment
+variables, and they must never touch each other's code or data:
+
+| | live | dev |
+|---|---|---|
+| tree | `worktrees/agentus` | `worktrees/agentus-dev` |
+| role | the instance the operator actually uses | where changes are made and tested |
+| HTTP port | `8787` | `8901` |
+| TLS port | `8443` | off (`AGENTUS_TLS_PORT=0`) |
+| data dir | `<tree>/packages/server/.data` (pinned by the launcher) | `/tmp/agentus-qa-account` |
+| launcher | `~/.hermes/cache/agentus/launch.py` | `scripts/dev.sh start\|stop\|status\|logs` |
+| hot reload | **no** — restart is explicit | yes (`tsx watch`) |
+
+**Why two trees.** `tsx watch` reloads the whole server on any server-source edit, and a
+reload SIGTERMs every child process — every live `hermes acp` session, including the one
+the operator is typing into. Editing server code in the live tree would therefore kill his
+conversation mid-turn (measured: page went white mid-turn, the turn died). The live tree
+runs **without** a watcher and is only moved by an explicit release
+([03-release.md](./03-release.md)). All edits happen in the dev tree.
+
+**The loop you repeat every round:**
+
+```
+change (dev tree)  →  verify (sweeps + suites)  →  promote (dev→live)  →  record (track + lessons)
+```
+
+Nothing merges to `main` or reaches the live instance without the operator saying so —
+the one exception is a strictly web-only change, and even that is bounded by what
+`promote` prints (see [03-release.md](./03-release.md)).
+
+## The five documents
+
+| doc | the question it answers |
+|---|---|
+| [01-dev-loop.md](./01-dev-loop.md) | How do I run a development instance, and how is its data kept separate from live? |
+| [02-verify.md](./02-verify.md) | How do I prove a change actually works — unit suites and real-browser sweeps? |
+| [03-release.md](./03-release.md) | How do I get a verified change onto the live instance, and how do I roll it back? |
+| [04-recover.md](./04-recover.md) | The instance is broken — where do I look, and how do I fix it safely? |
+| [05-agent-driven.md](./05-agent-driven.md) | How do I hand a task on this repo to a coding agent, and review what it does? |
+
+## Related in-repo docs
+
+- [`docs/transcript-folding.md`](../transcript-folding.md) — how one reply becomes rows,
+  pages and bubbles (the store's row semantics).
+- [`docs/android-notify-contract.md`](../android-notify-contract.md) — the server↔phone
+  notification contract.
+- [`docs/refs/`](../refs/) — notes on the reference UIs and ACP rules this project borrows
+  behaviour from.
+- [`README.md`](../../README.md) — install, environment variables, login, TLS and the
+  public tunnel.