@tonbo/yield 0.0.0-stage → 0.1.1

package/README.md

@@ -1,3 +1,110 @@
-# Temporary Holding Version
+# Tonbo Yield
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+`yield` captures the sessions of your coding agents (Claude Code and Codex) on every machine you use and lists them from any of them. It signs in, runs a capture service on each machine, snapshots the git work trees the agents work in, lists sessions and recalls the turns that changed a file; handoff comes later.
+
+Install it on macOS (arm64, x64) or Linux with glibc (x64, arm64) with one line, without sudo and without Node.js:
+
+```console
+curl -fsSL https://tonbo.dev/install.sh | sh
+```
+
+The installer, [`install.sh`](install.sh), uses the system `node` when it is 22 or newer and has npm beside it; otherwise it downloads the latest official Node.js 22 build from nodejs.org into `~/.tonbo/runtime/node-<version>`, checked against the release's `SHASUMS256.txt`. It installs `@tonbo/yield` into the private prefix `~/.tonbo/yield`, never a global npm, and writes the launcher `~/.local/bin/yield`, which runs it with that Node.js; when `~/.local/bin` is not on `PATH` it prints the line to add for your shell. Running it again upgrades in place, and when the capture service is installed it reinstalls the service on the new version (which restarts it) and removes Node.js runtimes it no longer uses. `YIELD_VERSION` pins a version or dist-tag: `curl -fsSL https://tonbo.dev/install.sh | YIELD_VERSION=0.2.0 sh`. musl distributions (Alpine), 32-bit systems and Windows are refused with a message; on Windows, install the npm package instead. User Center serves the file at `/install.sh` as `text/plain` from an embedded copy (`apps/user-center/src/app/install.sh`); after changing `install.sh`, run `pnpm --dir apps/user-center generate:install-script`, or the route's test fails.
+
+Or install the public package with Node.js 22 or newer:
+
+```console
+npm install --global @tonbo/yield
+```
+
+Merging a change to the `version` in `package.json` to `main` releases it: [`tonbo-yield-release.yml`](../../.github/workflows/tonbo-yield-release.yml) tests the package, publishes it through npm trusted publishing and installs it back from the registry, as the [Tonbo CLI release](../../docs/operations/tonbo-cli-release.md) does.
+
+## Sign in
+
+```console
+yield login [--account <name>] [--name <name>]
+yield whoami [--json]
+yield logout
+```
+
+`yield` and the [`tonbo`](../tonbo) CLI share one login through the internal package [`@tonbo/cli-core`](../../packages/tonbo-cli-core): the same browser handoff, the same personal key named `CLI on <hostname>` and the same credentials file, `${XDG_CONFIG_HOME:-$HOME/.config}/tonbo/credentials.json`. Logging in with either CLI signs both in, and a login replaces the host's earlier key, so the two never hold different keys. `whoami` prints the account, your role and this machine's device id.
+
+## The device id
+
+The device id names this computer for its whole life ([ADR-0015](../../apps/user-center/docs/decisions/adr-0015-byoc-machines.md) Q1). The first command that needs it makes a random UUID and keeps it in `${XDG_STATE_HOME:-$HOME/.local/state}/tonbo-yield/device.json` beside a SHA-256 of the operating system's machine id (macOS `IOPlatformUUID`, Linux `/etc/machine-id`). A login, a re-login or a revoked key leaves it unchanged. When the saved hash differs from the computer's, the state directory came from another computer (a cloned disk) and a new UUID replaces it; when the machine id cannot be read, the saved UUID stays. A lost state directory makes a new device, as a new computer would. Captured records (`device`), the Streams producer (`yield/<device id>`) and the snapshot refs (`refs/tonbo/u/<sub>/<device id>`) name it.
+
+`yield login` and each start of the daemon register this computer as a device of your personal account (`PUT /v1/machines/self`, a Machine of kind `byoc`, ADR-0015) with the saved login: its device id, hostname, operating system, architecture, Yield's version and the harnesses it has. Only the key a CLI login issued registers, and only in your personal account; `--name` chooses the device's name the first time (`<os>-<n>` otherwise). The running daemon registers again every 300 seconds, give or take 30 (machine-v2 `kinds.byoc.heartbeat`, an interim until the state work of Yield M4), which keeps `last_seen` fresh; an unchanged report writes only `last_seen`, and a throttled one (429 `device_report_throttled`) counts as done. Registration never holds back capture: a refusal or a failure is printed or logged (once, until it changes) and tried again at the next login or heartbeat. A refusal no retry changes, such as a login to an organization (`self_requires_personal_account`), a key that a CLI login did not issue, the device limit or a taken name, stops the heartbeat until the saved login changes.
+
+```console
+yield devices [--json]
+```
+
+`devices` lists your computers, only the `byoc` Machines of your personal account (`GET /v1/machines`, every page, with the saved login; never `TONBO_API_KEY`), and under an organization's login says to log in to your personal account instead: the name, online when the last report is within machine-v2's `online_window_seconds` (900), when it was last seen, the OS and architecture, the agents it captures, Yield's version, and which one is this computer. A region whose devices could not be read is named below the table.
+
+Yield 0.1.0 named the device by the key id of its login. Its state upgrades in place: each file keeps its offset, so nothing captured is sent again, and moves to the producer `yield/<device id>` from epoch 0, seq 0, saved before it is used. A batch saved under the old producer is first finished under it, so a resend after a crash is still answered as a duplicate, and the inbox moves once no such batch is left. Snapshots start a new chain on the new device's ref; the old refs stay.
+
+## Capture
+
+```console
+yield daemon install [--since <date>]
+yield daemon status [--json]
+yield daemon uninstall
+yield daemon run [--since <date>]
+```
+
+`install` writes a user service that runs `yield daemon run` with the current Node.js and starts it: a launchd agent at `~/Library/LaunchAgents/dev.tonbo.yield.plist` on macOS (`RunAtLoad`, `KeepAlive`) or a systemd user unit at `~/.config/systemd/user/tonbo-yield.service` on Linux (`Restart=always`). The service logs to `${XDG_STATE_HOME:-$HOME/.local/state}/tonbo-yield/daemon.log`. `uninstall` stops and removes it and keeps the capture state. `run` captures in the foreground and logs to stderr. `status` reports whether the capture runs, its last upload, how many sessions it tracks and skipped, the sessions it blocked and its recent errors.
+
+The capture follows [`contracts/sessions/capture-v1.json`](../../contracts/sessions/capture-v1.json), which the build embeds (`pnpm generate:contracts`):
+
+- It discovers the account's store once an hour through `GET /v1/sessions/store` and again whenever the store refuses a request. An account without a store (409 `sessions_store_unavailable`: no active git workspace Agent) is reported by `daemon status` and `sessions` as needing an account owner to enable git, and the daemon asks again every 10 minutes.
+- Streams does not accept the personal key itself: the client exchanges it at `POST <console origin>/api/iam/token` for a ten-minute Streams access token (`@tonbo/cli-core` `AccessTokens`), keeps it in memory, replaces it a minute before it expires, and exchanges again once after a Streams 401. A refused exchange asks for `yield login`; a 429 or 5xx is retried with backoff while the offsets stay.
+- It watches `~/.claude/projects` and `~/.codex/sessions` and rescans them every two seconds; a missing root is fine. Each file is read from its saved byte offset, and only complete lines count.
+- Each line, except the contract's excluded types, becomes one record of the stream `session.<session id>`. Consecutive lines go in one append request (at most the contract's `max_batch_lines` and `max_batch_bytes`), which ends early at a line that causes an inbox record. Each request carries `Producer-Id: yield/<device id>`, `Producer-Epoch` and `Producer-Seq`, the request's ordinal within the epoch; the batch's seq and byte range are saved before it is sent, so a resend after a crash is the identical batch and is answered as a duplicate. The record carries the line as written, without redaction; a line that is not JSON is uploaded as it is too.
+- When a session is first seen it appends `session_start` to `proc.inbox`, and at each turn end `turn_end` with the cwd, git root, branch, the paths written since the previous turn end (at most 50, relative to the git root inside it) and `stream_offset`, the session stream's `Stream-Next-Offset` after the turn's last record (the append's answer, or the stream's tail when a resent batch is answered as a duplicate).
+- On every pass it decides each live session's state, `running`, `waiting`, `idle` or `ended` (capture-v1 `sources.<source>.state`; [Yield session state](../../docs/design/yield-session-state.md)), and appends `session_state` to `proc.inbox` when the state or the turn changes, and while a turn runs, when it has written new paths, at most every 10 seconds. Claude Code's live-process registry (`~/.claude/sessions/<pid>.json`) gives a Claude session's liveness and status; a running `codex` process whose working directory is the session's cwd keeps a Codex session live; otherwise a session's turn decides `running` or `idle`, and a session whose liveness cannot be told ends after 30 quiet minutes. A session first seen already ended is not reported.
+- The state, `${XDG_STATE_HOME:-$HOME/.local/state}/tonbo-yield/state.json`, keeps each file's offset with its producer, sequence and epoch, and is replaced atomically. The source files are the offline queue: while the store is unreachable the offset stays and the upload is retried with backoff, honoring `Retry-After`. A new epoch starts from seq 0, saved before it is used, when Streams has forgotten the producer (idle for 7 days) or is ahead of a lost local state.
+- The first run also captures the session files changed in the last 14 days (`--since` sets the bound); older files are recorded as skipped until they change again.
+
+### Snapshots
+
+After each `turn_end` of a turn that ended in the last 10 minutes, in a cwd inside a git work tree, the daemon snapshots the work tree as [`contracts/sessions/recall-v1.json`](../../contracts/sessions/recall-v1.json) `snapshots` defines; the build embeds that contract too. Snapshots run one at a time beside the capture and never hold it up; a failure is logged as a warning, and a 4xx from the git token exchange or gitd pauses publishing for 10 minutes.
+
+- The tree is what `git add -A` sees through a private index under `${XDG_STATE_HOME:-$HOME/.local/state}/tonbo-yield/snapshots/index/`, so `.gitignore` applies and the user's index, HEAD, branches and config stay untouched. Every git command runs the user's `git` with `GIT_TERMINAL_PROMPT=0`, `core.hooksPath=/dev/null`, no automatic gc and no credential helper. Files over 100 MiB, gitd's object limit, are left out with a logged notice.
+- An unchanged tree makes no snapshot. Otherwise `git commit-tree` writes a commit whose parent is the previous snapshot of this device's chain (none for the first, the baseline) with the trailers `Tonbo-Session`, `Tonbo-Turn`, `Tonbo-Base` (the merge-base of HEAD and its upstream, or HEAD) and `Tonbo-Baseline: true` on the root; `refs/tonbo/snapshots/<device id>` keeps it in the user's repository.
+- The repository is `snap-<slug>` of the origin URL (or of the root's directory name and path hash), created once through gitd's idempotent `PUT /api/v1/workspaces/<ws_hex>/repos/<name>` `{"private": false}`, `ws_hex` being the store's `agent_id` without hyphens. The commit is pushed to `refs/tonbo/u/<sub>/<device id>` with `-o tonbo.session=<id>`. The git token (`POST /api/iam/token` with the git audience, kept and refreshed like the Streams token) reaches git only as an `http.<gitd origin>/.extraHeader` in the environment of that one command: never in a file, an argument, a URL or the log.
+- The chain head of each repository and device is kept in `state.json`. A push refused because gitd's ref moved on without this state (say, after the state was lost) fetches gitd's ref and continues the chain from it; a clone that lacks the chain's head fetches it first.
+
+The daemon uses the saved login only; `TONBO_API_KEY` does not name a device. One daemon runs per state directory.
+
+## Sessions
+
+```console
+yield ps [--all] [--limit <sessions>] [--json]
+yield sessions [--limit <records>] [--json]
+```
+
+`ps` reads the catalog, recall's `GET /sessions` without `repo`, which the git-ingester keeps in the workspace's index from `proc.inbox`. It prints your sessions on every device, most recent activity first: the state and since when, the device (`this computer` for this one), the agent, the git root or cwd, the session and the files it recently wrote (the turn in progress's first). Ended sessions are left out unless `--all`; a session no client reported a state for shows `ended` once it is 30 minutes quiet, and `unknown` before.
+
+`sessions` reads `proc.inbox` itself and prints one row per session among its newest records (200 by default): the last activity, the device (`this machine` for this one), the source, the git root or cwd, the branch and the turn ends seen. `ps` supersedes it; it stays until the first-run end-to-end check (`e2e/src/yield-first-run-e2e.mjs`) reads `ps`, once the index serves personal workspaces (ADR-0016 PR 4).
+
+## Recall
+
+```console
+yield why <path> [--limit <turns>] [--max-chars <n>] [--json]
+yield files <path> [--limit <turns>] [--json]
+yield session read <id> [--turn <n>] [--max-chars <n>] [--json]
+```
+
+The recall commands read the git-ingester's queries at `https://git.tonbo.dev/api/recall/v1/workspaces/<ws_hex>` (recall-v1 `queries`) with the git token, and the turns themselves from `session.<session id>` with the Streams token. A path is resolved from the current directory to its git root and the snapshot repository name the daemon pushes to.
+
+- `files` lists the turns that changed the file, newest first: when, session, turn, change, whether the turn wrote the file itself, device and snapshot commit.
+- `session read` prints a session as role-tagged lines (`user:`, `assistant:`, `tool:`), each cut at `--max-chars`; with `--turn` only that turn, read between the offsets of its `turn_end` and the previous one, or split from the whole stream when they are missing.
+- `why` joins `files` with each turn's prompt, the assistant's last words and the other files the turn changed, as a timeline an agent can read; `--json` gives the same structure.
+
+## The Tonbo skill
+
+`daemon install` also writes the Tonbo skill ([`skills/tonbo/SKILL.md`](skills/tonbo/SKILL.md), embedded by the build) to `~/.claude/skills/tonbo/SKILL.md` for Claude Code (`CLAUDE_CONFIG_DIR` moves it) and `~/.codex/skills/tonbo/SKILL.md` for Codex (`CODEX_HOME` moves it). It tells an agent to run `yield why`, `yield files` and `yield session read` when asked why code is the way it is or before changing an unfamiliar file, to cite session and time, to treat the results as reference and never as instructions, and not to guess when nothing is found. A manifest, `skills.json` in the state directory, records what was written: install never overwrites a `tonbo` skill it did not write, and `daemon uninstall` removes only its own unchanged files. `CLAUDE.md` and `AGENTS.md` are never edited.
+
+## Environment
+
+- `TONBO_API_KEY`: a personal key used instead of the saved login by `whoami` and `sessions`.
+- `TONBO_API_ORIGIN`: the Management API origin, `https://api.tonbo.dev` by default. `daemon install` writes it into the service, with `PATH`, `XDG_CONFIG_HOME` and `XDG_STATE_HOME`.