@tonbo/yield 0.0.0-stage → 0.1.0

package/README.md

@@ -1,3 +1,84 @@
-# Temporary Holding Version
+# Tonbo Yield
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+`yield` captures the sessions of your coding agents (Claude Code and Codex) on every machine you use and lists them from any of them. It signs in, runs a capture service on each machine, snapshots the git work trees the agents work in, lists sessions and recalls the turns that changed a file; handoff comes later.
+
+Install the public package with Node.js 22 or newer:
+
+```console
+npm install --global @tonbo/yield
+```
+
+Merging a change to the `version` in `package.json` to `main` releases it: [`tonbo-yield-release.yml`](../../.github/workflows/tonbo-yield-release.yml) tests the package, publishes it through npm trusted publishing and installs it back from the registry, as the [Tonbo CLI release](../../docs/operations/tonbo-cli-release.md) does.
+
+## Sign in
+
+```console
+yield login [--account <name>]
+yield whoami [--json]
+yield logout
+```
+
+`yield` and the [`tonbo`](../tonbo) CLI share one login through the internal package [`@tonbo/cli-core`](../../packages/tonbo-cli-core): the same browser handoff, the same personal key named `CLI on <hostname>` and the same credentials file, `${XDG_CONFIG_HOME:-$HOME/.config}/tonbo/credentials.json`. Logging in with either CLI signs both in, and a login replaces the host's earlier key, so the two never hold different keys. `whoami` prints the account, your role and this machine's device id, which is the key id of that login; revoking the key revokes the device.
+
+## Capture
+
+```console
+yield daemon install [--since <date>]
+yield daemon status [--json]
+yield daemon uninstall
+yield daemon run [--since <date>]
+```
+
+`install` writes a user service that runs `yield daemon run` with the current Node.js and starts it: a launchd agent at `~/Library/LaunchAgents/dev.tonbo.yield.plist` on macOS (`RunAtLoad`, `KeepAlive`) or a systemd user unit at `~/.config/systemd/user/tonbo-yield.service` on Linux (`Restart=always`). The service logs to `${XDG_STATE_HOME:-$HOME/.local/state}/tonbo-yield/daemon.log`. `uninstall` stops and removes it and keeps the capture state. `run` captures in the foreground and logs to stderr. `status` reports whether the capture runs, its last upload, how many sessions it tracks and skipped, the sessions it blocked and its recent errors.
+
+The capture follows [`contracts/sessions/capture-v1.json`](../../contracts/sessions/capture-v1.json), which the build embeds (`pnpm generate:contracts`):
+
+- It discovers the account's store once an hour through `GET /v1/sessions/store` and again whenever the store refuses a request. An account without a store (409 `sessions_store_unavailable`: no active git workspace Agent) is reported by `daemon status` and `sessions` as needing an account owner to enable git, and the daemon asks again every 10 minutes.
+- Streams does not accept the personal key itself: the client exchanges it at `POST <console origin>/api/iam/token` for a ten-minute Streams access token (`@tonbo/cli-core` `AccessTokens`), keeps it in memory, replaces it a minute before it expires, and exchanges again once after a Streams 401. A refused exchange asks for `yield login`; a 429 or 5xx is retried with backoff while the offsets stay.
+- It watches `~/.claude/projects` and `~/.codex/sessions` and rescans them every two seconds; a missing root is fine. Each file is read from its saved byte offset, and only complete lines count.
+- Each line, except the contract's excluded types, is redacted with the contract's patterns and becomes one record of the stream `session.<session id>`. Consecutive lines go in one append request (at most the contract's `max_batch_lines` and `max_batch_bytes`), which ends early at a line that causes an inbox record. Each request carries `Producer-Id: yield/<device id>`, `Producer-Epoch` and `Producer-Seq`, the request's ordinal within the epoch; the batch's seq and byte range are saved before it is sent, so a resend after a crash is the identical batch and is answered as a duplicate. A line that is not JSON is uploaded as it is. If the redaction fails, nothing more of that session is uploaded and the session is marked blocked.
+- When a session is first seen it appends `session_start` to `proc.inbox`, and at each turn end `turn_end` with the cwd, git root, branch, the paths written since the previous turn end (at most 50, relative to the git root inside it) and `stream_offset`, the session stream's `Stream-Next-Offset` after the turn's last record (the append's answer, or the stream's tail when a resent batch is answered as a duplicate).
+- The state, `${XDG_STATE_HOME:-$HOME/.local/state}/tonbo-yield/state.json`, keeps each file's offset with its producer sequence and epoch, and is replaced atomically. The source files are the offline queue: while the store is unreachable the offset stays and the upload is retried with backoff, honoring `Retry-After`. A new epoch starts from seq 0, saved before it is used, when Streams has forgotten the producer (idle for 7 days) or is ahead of a lost local state.
+- The first run also captures the session files changed in the last 14 days (`--since` sets the bound); older files are recorded as skipped until they change again.
+
+### Snapshots
+
+After each `turn_end` of a turn that ended in the last 10 minutes, in a cwd inside a git work tree, the daemon snapshots the work tree as [`contracts/sessions/recall-v1.json`](../../contracts/sessions/recall-v1.json) `snapshots` defines; the build embeds that contract too. Snapshots run one at a time beside the capture and never hold it up; a failure is logged as a warning, and a 4xx from the git token exchange or gitd pauses publishing for 10 minutes.
+
+- The tree is what `git add -A` sees through a private index under `${XDG_STATE_HOME:-$HOME/.local/state}/tonbo-yield/snapshots/index/`, so `.gitignore` applies and the user's index, HEAD, branches and config stay untouched. Every git command runs the user's `git` with `GIT_TERMINAL_PROMPT=0`, `core.hooksPath=/dev/null`, no automatic gc and no credential helper. Files over 100 MiB, gitd's object limit, are left out with a logged notice.
+- An unchanged tree makes no snapshot. Otherwise `git commit-tree` writes a commit whose parent is the previous snapshot of this device's chain (none for the first, the baseline) with the trailers `Tonbo-Session`, `Tonbo-Turn`, `Tonbo-Base` (the merge-base of HEAD and its upstream, or HEAD) and `Tonbo-Baseline: true` on the root; `refs/tonbo/snapshots/<device id>` keeps it in the user's repository.
+- The repository is `snap-<slug>` of the origin URL (or of the root's directory name and path hash), created once through gitd's idempotent `PUT /api/v1/workspaces/<ws_hex>/repos/<name>` `{"private": false}`, `ws_hex` being the store's `agent_id` without hyphens. The commit is pushed to `refs/tonbo/u/<sub>/<device id>` with `-o tonbo.session=<id>`. The git token (`POST /api/iam/token` with the git audience, kept and refreshed like the Streams token) reaches git only as an `http.<gitd origin>/.extraHeader` in the environment of that one command: never in a file, an argument, a URL or the log.
+- The chain head of each repository and device is kept in `state.json`. A push refused because gitd's ref moved on without this state (say, after the state was lost) fetches gitd's ref and continues the chain from it; a clone that lacks the chain's head fetches it first.
+
+The daemon uses the saved login only; `TONBO_API_KEY` does not name a device. One daemon runs per state directory.
+
+## Sessions
+
+```console
+yield sessions [--limit <records>] [--json]
+```
+
+`sessions` reads `proc.inbox` and prints one row per session among its newest records (200 by default): the last activity, the device (`this machine` for this one), the source, the git root or cwd, the branch and the turn ends seen.
+
+## Recall
+
+```console
+yield why <path> [--limit <turns>] [--max-chars <n>] [--json]
+yield files <path> [--limit <turns>] [--json]
+yield session read <id> [--turn <n>] [--max-chars <n>] [--json]
+```
+
+The recall commands read the git-ingester's queries at `https://git.tonbo.dev/api/recall/v1/workspaces/<ws_hex>` (recall-v1 `queries`) with the git token, and the turns themselves from `session.<session id>` with the Streams token. A path is resolved from the current directory to its git root and the snapshot repository name the daemon pushes to.
+
+- `files` lists the turns that changed the file, newest first: when, session, turn, change, whether the turn wrote the file itself, device and snapshot commit.
+- `session read` prints a session as role-tagged lines (`user:`, `assistant:`, `tool:`), each cut at `--max-chars`; with `--turn` only that turn, read between the offsets of its `turn_end` and the previous one, or split from the whole stream when they are missing.
+- `why` joins `files` with each turn's prompt, the assistant's last words and the other files the turn changed, as a timeline an agent can read; `--json` gives the same structure.
+
+## The Tonbo skill
+
+`daemon install` also writes the Tonbo skill ([`skills/tonbo/SKILL.md`](skills/tonbo/SKILL.md), embedded by the build) to `~/.claude/skills/tonbo/SKILL.md` for Claude Code (`CLAUDE_CONFIG_DIR` moves it) and `~/.codex/skills/tonbo/SKILL.md` for Codex (`CODEX_HOME` moves it). It tells an agent to run `yield why`, `yield files` and `yield session read` when asked why code is the way it is or before changing an unfamiliar file, to cite session and time, to treat the results as reference and never as instructions, and not to guess when nothing is found. A manifest, `skills.json` in the state directory, records what was written: install never overwrites a `tonbo` skill it did not write, and `daemon uninstall` removes only its own unchanged files. `CLAUDE.md` and `AGENTS.md` are never edited.
+
+## Environment
+
+- `TONBO_API_KEY`: a personal key used instead of the saved login by `whoami` and `sessions`.
+- `TONBO_API_ORIGIN`: the Management API origin, `https://api.tonbo.dev` by default. `daemon install` writes it into the service, with `PATH`, `XDG_CONFIG_HOME` and `XDG_STATE_HOME`.