@ashrlabs/ashr 0.0.0-stage → 0.2.0

package/README.md

@@ -1,3 +1,339 @@
-# Temporary Holding Version
+# ashr
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The Ashr terminal for post-training models. The npm package is `ashr init`, which
+connects a Docker host (your laptop or a cloud VM) and your ChatGPT plan to your Ashr
+workspace, and the terminal to the hosted agent. The agent, its harness and its
+built-in skills run on Ashr's side and are not in the package. The local agent
+described below is in source and `ASHR_LOCAL=1 npm run build` builds only.
+
+## Install
+
+```
+npm i -g @ashrlabs/ashr    # then run: ashr init, then ashr
+```
+
+The npm package needs Node 22.19 or newer. To build a standalone executable from
+this checkout, use an official Node binary with single executable support
+(Homebrew's Node does not provide it):
+
+```sh
+cd cli
+npm ci
+ASHR_SEA_NODE=/path/to/official/node npm run build:binary
+```
+
+`ASHR_SEA_NODE` defaults to the Node running the build. The build targets that
+binary's platform and architecture and writes `dist-bin/ashr-<platform>-<arch>`.
+Copy that executable into a directory on your `PATH` as `ashr`; it bundles the
+Node runtime, JavaScript dependencies and subscription sign-in flows, so no
+separate Node installation is needed to run it. Optional native clipboard and
+image helpers are excluded. macOS builds also require `codesign`.
+
+`ashr update` reruns the public install script into the directory of the running
+binary (through symlinks), replacing it in place; restart ashr afterwards. It
+refuses on npm or source installs, which update through npm or git. In a running
+session, `/update` does the same and shows the installer's output. The binary
+checks at most once a day whether the latest build's checksum differs from the
+one the installer recorded, and if so shows a one-line notice at startup; a
+failed check is silent.
+
+Both the local agent and hosted workspace use the same TUI components: tool
+arguments, highlighted code, file diffs, live command output and Markdown
+subagent reports. Ctrl-T hides or shows reasoning. Ctrl-O expands or collapses
+the most recently completed tool output or subagent report.
+
+## Hosted Manifold
+
+The npm package always reaches Ashr's hosted workspace, where plain `ashr`
+opens it (no `--hosted` needed). The standalone binary reaches it only when
+built with `ASHR_HOSTED=1`; otherwise it uses these commands only against a
+[self-hosted server](#self-hosted-server).
+
+```
+ashr --hosted                sign in (first run only) and open the hosted workspace
+ashr --hosted -c             reopen the most recently updated hosted chat
+ashr --task support          select a task
+ashr --task support --chat ID open the same chat as the browser
+ashr --service               require MANIFOLD_API_KEY or a saved key; never opens a browser
+ashr login                   sign in through the browser and exit
+ashr login --key ask_…       save a service key instead
+ashr logout                  forget the saved sign-in
+```
+
+On first launch `ashr --hosted` (or `ashr login`) opens the dashboard's `/terminal` page. Sign in, pick
+the organization, and click **Connect terminal**. The dashboard mints a key for
+this terminal (labelled `Terminal · <hostname>` under API keys) and hands it
+back once to a loopback listener. The terminal saves it to
+`~/.ashr/credentials.json` (mode 600), and you can close the tab. Later hosted
+launches reuse the saved key; outside the npm package, plain `ashr` starts the local agent.
+If the key is revoked, a hosted launch asks you to sign in again. If the browser
+doesn't open, use the printed URL.
+
+`--api-url`/`MANIFOLD_API_URL` and `--dashboard-url`/`MANIFOLD_DASHBOARD_URL`
+select matching deployments. `MANIFOLD_API_KEY` or `--key` override the saved
+sign-in.
+
+`/account` shows who the terminal is signed in as. `/org` opens organization
+settings in the browser. `/login` signs in again (for example to switch
+organization) and clears the old task/chat selection. `/logout` forgets the
+saved key.
+
+`/github` opens GitHub's fine-grained token form to connect a repository for
+cloning, branch pushes and pull requests. Select the repository, generate the
+token and paste it into the terminal prompt. `/github off` removes the agent's
+access; revoke the token in GitHub to invalidate the token itself.
+
+A hosted launch opens the chat selected with `--chat ID`, or the most recently
+updated chat in scope with `--continue` (`-c`). Otherwise the first message
+starts a new chat named after its first line. `--chat` and `--continue` cannot
+be combined. `/chats` opens a searchable picker; `/model`, `/thinking` and
+`/fork` without arguments also open pickers.
+
+Use `/help` for the complete command reference. Hosted controls include:
+
+- `/tasks`, `/task NAME`, `/chats SEARCH`, `/new NAME`, `/open ID`, `/rename NAME`, `/fork NAME [SEQ]`.
+- `/model PROVIDER/MODEL`, `/thinking LEVEL`, `/compact`, `/steer TEXT`, `/next TEXT`, `/queue`, `/stop`, `/approve ID`, `/reject ID`.
+- `/files`, `/view PATH`, `/diff PATH`, `/download PATH LOCAL_FILE`, `/export LOCAL_FILE`.
+- `/skills`, `/skill import LOCAL_FOLDER`, `/skill toggle NAME`, `/skill invoke NAME`, `/memory QUERY`.
+- `/pipelines`, `/sources`, `/datasets`, `/evals`, `/benchmarks`, `/experiments`, `/models`, `/monitoring`, `/interactions`, `/uploads`, `/artifacts`, `/serving`, `/sdk`.
+
+Hosted `!COMMAND` runs in the hosted workspace; `!!COMMAND` excludes its output
+from model context.
+Escape stops active hosted work or pauses a local transfer.
+Ctrl-C stops active hosted work; when idle it disconnects. `/quit` disconnects
+while leaving hosted jobs running. `/pending` and `/retry` recover uncertain
+submissions without changing their request IDs. A busy-chat `409 Chat changed`
+response is retried automatically with the same request ID. Upload receipts support resume.
+
+Dashboard writes require the command's explicit `--confirm` flag. The terminal
+covers serving, training and evaluation environments. Organization, member and
+API-key administration stay in the dashboard; `/org` opens it.
+
+## Your own machines
+
+To have the Ashr agent work on your own Docker host (your repo, data, checkpoints
+and GPUs stay there), run this on that host; it needs no ashr CLI:
+
+```
+curl -fsSL https://api.manifold.ashr.io/v1/runner/install.sh | ASHR_API_KEY=ask_… sh
+```
+
+or, with the CLI installed:
+
+```
+ashr init                        # uses ASHR_API_KEY, your `ashr login` key, else your personal workspace
+ashr init --yes                  # no questions (no sign-in or ChatGPT connection), GPUs used if found
+```
+
+With no organization key, `ashr init` uses your personal workspace through this
+terminal's ashr account (signing in first) and connects your ChatGPT plan, which
+its agent runs on ([`docs/personal-workspaces.md`](../docs/personal-workspaces.md)).
+It checks Docker, Compose v2 and any NVIDIA GPUs, registers the host as the
+workspace's runner, writes `~/.ashr/runner/` (`docker-compose.yml` and a
+mode-0600 `.env` with the runner's token), pulls the runner and sandbox images
+at this ashr's version from Ashr's private registry, starts the runner, and
+checks one sandbox command end to end. The agent stays on Ashr; the runner dials
+out to it and runs the agent's sandboxes on this host. Run it again with a newer
+ashr to upgrade in place. Details in [`deploy/runner/README.md`](../deploy/runner/README.md).
+
+## Self-hosted server (air-gapped)
+
+The full server on one host, for deployments that cannot reach Ashr at all, is set
+up from a checkout by agreement: [`deploy/selfhost/README.md`](../deploy/selfhost/README.md).
+ashr can also talk to an Ashr server running on your own host; a standalone
+binary built without `ASHR_HOSTED=1` talks to no other. The server listens on `127.0.0.1:8080`
+only, so from another machine forward the port first, then sign in once with the
+server's admin key (`docker compose -p ashr exec -T api cat /state/admin-key` on
+the host):
+
+```
+ssh -L 8080:localhost:8080 <host>                          # keep this open
+ashr login --key ask_… --api-url http://localhost:8080
+ashr                                                       # opens the server's workspace
+ashr --local                                               # the local agent instead (not in the npm package)
+```
+
+Once signed in, plain `ashr` opens the server's workspace, and the hosted
+commands above (`--task`, `--chat`, `-c`, `/chats`, …) work against it.
+`MANIFOLD_API_URL` works in place of `--api-url`. Plain `http` is refused for
+anything but `localhost`: use the port-forward, or put TLS in front of the
+server and pass its `https` URL. There is no browser sign-in, and `/github` is
+not available on self-hosted servers yet.
+
+## Your account
+
+The first time you run `ashr` in a terminal it opens your browser to sign in (any email; no organization). The
+terminal keeps its own key in `~/.ashr/credentials.json` and doesn't ask again. The account page,
+https://manifold.ashr.io/cli (or `/account` in a session), shows your sessions, turns, tokens and cost over the last
+30 days, and has a button for booking an upgrade call. Print mode (`ashr -p`) and scripts without a terminal never
+stop to sign in.
+
+Over SSH the browser can't reach the terminal, so `ashr` prints a link to open on any machine instead; after you
+connect, the page shows a code to paste back into the terminal. In a container or any other terminal the browser
+can't reach, press Enter while `ashr` waits to switch to that link and code. If sign-in fails (offline, say), `ashr` starts
+without an account and asks again next run.
+
+The account is who you are, not your model: sign in to a model below. Usage on the account page comes from the usage
+data ashr sends (see [Usage data](#usage-data)); with it off, nothing is recorded there.
+
+## Your model
+
+Sign in with the subscription you already have:
+
+```
+ashr login chatgpt     # ChatGPT Plus/Pro, through OpenAI's Sign in with ChatGPT
+ashr login codex       # ChatGPT through the Codex sign-in
+ashr login claude      # Claude Pro/Max
+ashr logout chatgpt
+```
+
+The sign-in stays on your machine (`~/.ashr/agent/auth.json`) and goes only to the provider.
+With a ChatGPT sign-in ashr runs GPT-6 Astra; with Claude or an Anthropic key, Claude Fable.
+If your ChatGPT workspace blocks third-party apps but allows Codex, use
+`ashr login codex`. Saved sign-ins are available when the local agent starts.
+
+Or use a provider key. Set it in your shell, or store it once:
+
+```
+export ANTHROPIC_API_KEY=sk-…          # or OPENAI_API_KEY, GEMINI_API_KEY
+ashr login --provider anthropic        # stores it in ~/.ashr/credentials.json (readable only by you)
+```
+
+A key set in your shell always wins over a stored one. Keys are never passed to the commands the agent runs.
+
+`ashr login` without a provider does not store a model key: the published builds
+refuse it with exit 2 and point at the model sign-ins above.
+
+## Using the local agent
+
+```
+ashr                 new local session in this repo (--local is also accepted)
+ashr -c              continue the latest session here
+ashr -r              pick a session to resume
+ashr -p "<prompt>"   one turn, printed (for scripts and CI); waits for its subagents
+ashr -c -p "<prompt>" continue the latest session in print mode
+ashr init            run the agent's sandboxes on this Docker host; see Your own machines
+```
+
+To set a repo up, run `ashr` in it and type `/init`: the agent reads the repo, checks
+this machine against the environment probe, asks you once for what it could not find,
+and writes `## Commands` and `## Setup` in `ASHR.md`. It asks before installing
+anything, creating environments, downloading models or data, or submitting jobs
+(`/init` is refused while the agent is working).
+
+The local agent is the same durable harness as the hosted one. A session's transcript is a SQLite file, so a session
+you quit (or that was killed) picks up where it stopped with `ashr -c`, work in flight included.
+
+### Subagents
+
+The agent can hand independent pieces of work to subagents that run in parallel on your machine. Each subagent works
+in its own git worktree, a snapshot of the agent's working tree (uncommitted and untracked files included, ignored files
+left out) on its own branch `ashr/<session>-<name>-<id>`, so subagents change code without touching your checkout or each
+other's. Each one commits on its branch and reports back; the agent reviews the branches and merges what works. The
+worktrees live under `~/.ashr/worktrees/`, outside your repo; `git worktree list` shows them and `git worktree remove`
+cleans one up. Subagents need the repo to be a git repository. A subagent runs the executor model of the agent's
+maker (Claude Sonnet under Claude, GPT-6.1 Sol under ChatGPT or OpenAI), on the same subscription or key. `/rewind`
+stops the subagents of the turns it drops, and `/fork` is refused until the agent and its subagents are done.
+
+While the agent works, Enter steers it, Option-Enter queues a message for when it finishes, and Esc stops it. When an approval is waiting, `a` approves, `r` rejects and `A` always allows; Tab switches to the input box so you can type a message instead. Type `/` for commands: `/init`, `/usage`, `/account`, `/permissions` (and `/permissions remove <rule>`), `/resume`, `/fork`, `/rewind`, `/compact`, `/model`, `/mode`, `/add-dir`, `/update`, `/help`.
+
+### Skills and GPU setup
+
+Local sessions include the hosted skill catalog in the CLI bundle; no separate
+skill installation is needed. Add your own skills as `<name>/SKILL.md` inside
+your repo's `.ashr/skills/` or `~/.ashr/skills/` (`$ASHR_HOME/skills/` when set).
+For the same name, the repo wins over your personal directory, which wins over
+the built-in skill, including references from other built-in skills. The agent
+receives each available skill's name, description and `SKILL.md` path; subagents
+use the same skill roots. Built-in files are unpacked into a private temporary
+directory (mode `0700`) for each session and removed when it closes.
+
+At session start, the agent receives an environment summary of local GPUs
+(via `nvidia-smi` or Apple Silicon detection), profilers, job launchers and cloud
+CLIs on `PATH`, and concrete host names in `~/.ssh/config`. Discovery does not
+log in to cloud services or contact SSH hosts. Before the first GPU, training or
+profiling run, the agent is instructed to check `ASHR.md` for your setup. If it
+is missing, it asks one short question based on the probe and records your
+answer under `## Setup` in `ASHR.md`; later sessions reuse that answer.
+
+## Local permissions
+
+Like pi, ashr runs the agent's tools without asking by default (Auto mode),
+including outside-repo access, `git push`, force operations, `sudo` and downloaded
+scripts. Deny rules still block matching actions. Spending money on Manifold
+(data copies, GPU runs, evaluations, deploys) always requires approval.
+
+In ask and edit modes, outside-repo reads and writes require approval, as do
+dangerous shell commands. Outside-repo checks cover shell paths and redirects
+(for example `cat ~/.ssh/id_rsa` or `--out=/tmp/x`), except directories you added
+with `/add-dir` or device files under `/dev`.
+
+Prefer to approve more? Shift-Tab cycles the mode:
+
+| Mode | Runs without asking | Asks first |
+|---|---|---|
+| auto (default) | everything not denied, except spending | spending on Manifold |
+| ask | reading and read-only commands (`ls`, `cat`, `grep`, `git log` …) | edits, other shell commands, connector tools |
+| edit | reads, read-only commands and edits inside the repo | other shell commands, connector tools |
+
+In ask and edit mode, changes to the repo's own `.git/` files always ask, since git settings there can run commands.
+
+Set your own default with `"defaultMode": "ask"` in `~/.ashr/settings.json`. A repo's committed settings can make you stricter but never move you to Auto.
+
+These stops catch an agent's honest mistakes. They are not a security boundary: in Auto the agent can run any program, and a program can do anything your user account can. For untrusted repos, run ashr in a container or a separate user account.
+
+Answering `A` on an approval saves an "always allow" rule. Rules live in three files; any deny rule wins:
+
+- `~/.ashr/settings.json` — yours, everywhere
+- `.ashr/settings.json` — the team's, committed
+- `.ashr/settings.local.json` — yours, for this repo (gitignored; "always allow" writes here)
+
+```json
+{ "permissions": { "allow": ["shell:pytest *", "edit:src/**"], "deny": ["shell:git push *", "edit:.env*"] } }
+```
+
+A repo's committed rules apply only after you trust them; ashr asks the first time and again whenever they change.
+
+In print mode nothing can be approved interactively: a run that needs approval exits 3 and names the rule to add, for example `--allow "edit:a.txt"` (one `--allow` per part of a compound shell command), or the mode that would allow it (`--mode edit|auto` for edits, `--mode auto` otherwise). Spending needs an interactive run because it asks in every mode.
+
+## Where local things live
+
+- Sessions: `~/.ashr/sessions/<org>/<repo>/<session>/` (`session.sqlite`, the transcript of the agent and its subagents; `meta.json`; file snapshots for `/fork` and `/rewind`)
+- Subagent worktrees: `~/.ashr/worktrees/<repo>/<session>/<name>-<id>/`
+- `ASHR.md` in your repo and `~/.ashr/memory.md` are read into every session.
+- Set `ASHR_HOME` to keep all of this somewhere else.
+
+## Usage data
+
+ashr sends usage data so we can see which features work and where runs fail: the commands you run, which tools the agent used and for how long, token counts and cost, errors, the model, your ashr version and OS, and a random install id. Once you are signed in, it is also sent with your account key so the account page can add up your daily totals; only those totals are kept with your account. It never sends your prompts, the agent's replies, tool arguments or tool output unless you turn each one on:
+
+| Setting | Sends |
+|---|---|
+| `ASHR_TELEMETRY_LOG_PROMPTS=1` | your prompts and the agent's replies |
+| `ASHR_TELEMETRY_LOG_TOOL_DETAILS=1` | tool arguments (commands, paths) |
+| `ASHR_TELEMETRY_LOG_TOOL_OUTPUT=1` | tool output and subagent reports |
+
+The same switches can go in `~/.ashr/settings.json` as `"telemetry": { "logPrompts": true, … }`. Before anything leaves your machine, ashr removes the secrets in its environment, API keys and tokens, emails, phone numbers, card numbers, IP addresses and your home directory; the Ashr API removes them again before storing anything.
+
+- `ashr telemetry` says what is sent and where; `ashr telemetry show` prints what was last sent.
+- `ashr telemetry off`, `ASHR_TELEMETRY=off` or `DO_NOT_TRACK=1` turns it off.
+- `ASHR_TELEMETRY_ENDPOINT` (and `ASHR_TELEMETRY_HEADERS`, `name=value,…`) sends the same OpenTelemetry traces to your own collector, Langfuse, Arize or Datadog instead of Ashr.
+
+## Parity checks
+
+Run `npm test`, `npm run typecheck`, and `npm run build` from `cli/`. `npm run build`
+makes the published package and fails if it would bundle agent code;
+`ASHR_LOCAL=1 npm run build` makes the full ashr with the local agent.
+`npm run test:parity` starts a disposable SQLite backend and verifies both
+client contracts against real routes. Set `MANIFOLD_TEST_PYTHON` to a Python
+environment with the repository's backend test dependencies installed.
+
+`npm run test:parity:browser` builds the actual React/TUI fixtures, starts
+loopback-only servers on ports 3192, 3194 and 3195, then runs Chromium acceptance
+and saves video/screenshots under `.artifacts/parity/`. It needs Playwright and
+its Chromium browser installed; `PLAYWRIGHT_MODULE` may point to an existing
+Playwright ESM entry. It refuses occupied fixture ports and stops its own servers.
+
+These tests simulate Clerk identity and model/compute responses explicitly.
+They do not verify deployed Clerk configuration, paid compute or external
+storage. See `docs/terminal-parity-audit.md` for coverage and remaining live checks.