hop2 0.9.0 → 0.10.0

package/AGENTS.md

@@ -0,0 +1,33 @@
+# AGENTS.md (hop2)
+
+Guidance for agents working on hop2.
+
+## Project shape
+- `hop` is the main Node.js CLI/server entry point (bin target).
+- `sessions.html` is the built-in session picker UI.
+- `hay/` contains the terminal sharing stack (server/cli/web).
+- `hay-web/` is the built web client output served by `hop`.
+
+## Local state & config
+- Default config dir is `~/.hop2` (override via `HOP_HOME`).
+- State files here should remain private (permissions matter).
+- Be careful with migrations: preserve backward compatibility where possible.
+
+## Change discipline
+- Keep changes focused and consistent with existing patterns.
+- Avoid unrelated refactors; minimize surface area.
+- Add concise comments only for non-obvious logic.
+- Update docs when user-visible behavior changes.
+
+## Build & test
+- Build web assets via `npm run build` (builds `hay` then syncs to `hay-web`).
+- Basic sanity check: `npm test` (runs `node --check hop`).
+- If you don’t run tests, say so and suggest likely commands.
+
+## UI work
+- Preserve existing visual language unless a redesign is intended.
+- Ensure layouts remain usable on desktop and mobile.
+
+## Server/CLI work
+- Prefer clear errors and non-zero exit codes on failures.
+- Keep security-sensitive behavior conservative and explicit.

package/INTEGRATION.md

@@ -1,92 +1,84 @@
-# Hop + Termshare Integration Plan
+# Hop + Hay Architecture
 
-This branch integrates [termshare](https://github.com/jzthree/termshare) to replace ttyd/tmux.
+This document describes the current Hop terminal architecture.
 
-## Benefits
+The historical `ttyd`/`tmux` backend has been removed from the active runtime.
+Hop now uses Hay, a local terminal-sharing stack built on WebSocket plus
+`node-pty`, with browser, CLI, and MCP clients all operating on the same
+session model.
 
-- **No ttyd/tmux dependencies** - Pure Node.js solution
-- **Built-in presence** - See who's connected, typing indicators
-- **Control modes** - Collaborative or locked single-controller
-- **No injection hacks** - Direct control over web UI
-- **Mobile-ready** - Responsive design with iOS-style keyboard built-in
-- **CLI client** - Connect from real terminals too
+## Current Stack
 
-## Architecture Change
-
-### Before (ttyd)
 ```
-Browser → Hop Auth Proxy → ttyd (per session) → tmux → shell
-                ↓
-        HTML/WebSocket injection for mobile keyboard
+Browser / Hay CLI / Hop MCP
+            |
+            v
+        Hop daemon
+            |
+            v
+     local hay-host process
+            |
+            v
+      Hay room manager
+            |
+            v
+         node-pty
+            |
+            v
+           shell
 ```
 
-### After (termshare)
-```
-Browser → Hop Auth Proxy → Termshare WebSocket → node-pty → shell
-                ↓
-        Native termshare web UI (mobile keyboard built-in)
-```
-
-## Implementation Steps
-
-### 1. Add Dependencies
-- [x] `node-pty-prebuilt-multiarch` - PTY handling
-- [x] `ws` - WebSocket server
-- [x] Copy termshare room/pty logic into hop (`termshare.js`)
-
-### 2. Remove ttyd/tmux Code
-- [x] Remove `spawn('ttyd', [...])` in `getOrCreateSession()`
-- [ ] Remove `HOP_TMUX_CONF`, `HOP_TMUX_WRAPPER` generation (still present but unused)
-- [ ] Remove `listTmuxSessions()`, `waitForTmuxSessionReady()` (still present but unused)
-- [ ] Remove ttyd dependency check (still present but unused)
-- [x] Remove keyboard HTML injection code (now using termshare's built-in)
-- [x] Remove `getIOSKeyboardInjection()` WebSocket patching (still present but unused)
-
-### 3. Add Termshare Integration
-- [x] Port `Room` and `RoomManager` classes from termshare
-- [x] Port `createPty` function
-- [x] Create termshare WebSocket handler at `/ws` endpoint
-- [x] Map hop sessions to termshare rooms
-
-### 4. Update Session Management
-```javascript
-// Before: activeSessions[name] = { ttyd: Process, port: number }
-// After:  activeSessions[name] = { room: Room }
-```
-✅ Completed
-
-### 5. Serve Termshare Web UI
-- [x] Build termshare web client (with mobile keyboard)
-- [x] Serve static files from hop (`termshare-web/` directory)
-- [x] Inject room parameter for session routing
-
-### 6. Keep Hop Features
-- [x] Authentication (TOTP + cookies)
-- [x] Cloudflare tunneling
-- [x] Session URL routing (`/s/<name>/`)
-- [x] Multi-user support
-
-## Key Files Changed
-
-- `hop` - Main server, updated `getOrCreateSession()` and WebSocket handler
-- `termshare.js` - New module with Room, RoomManager, createPty
-- `termshare-web/` - Built termshare web client with mobile keyboard
-
-## Testing
-
-1. Start hop: `node hop`
-2. Open browser to tunnel URL
-3. Verify:
-   - [ ] Login with TOTP works
-   - [ ] Terminal session works
-   - [ ] Multiple clients see same session
-   - [ ] Presence indicators show
-   - [ ] Control modes work
-   - [ ] Mobile UI works without injection
-
-## Migration Notes
-
-- Sessions are not persistent (by design) - shell exits when all clients disconnect
-- tmux scrollback replaced by termshare's output buffer (200KB)
-- iOS keyboard injection replaced by termshare's built-in mobile keyboard
-- The old ttyd/tmux code paths are still present but no longer used; can be removed in a cleanup pass
+Key properties:
+
+- No `ttyd` dependency
+- No `tmux` dependency
+- Shared session model across browser, CLI, and MCP
+- Session-aware audit logging and history isolation
+- Native mobile UI and virtual keyboard support
+- Local host process survives hop daemon restarts
+
+## Main Components
+
+- `hop`
+  - user-facing CLI and daemon
+  - auth, Cloudflare tunnel management, session metadata, workspace handling
+  - serves the built Hay web client from `hay-web/`
+- `scripts/hay-host.js`
+  - persistent local host process for active Hay rooms
+  - isolates PTY/runtime ownership from the daemon lifecycle
+- `hay/apps/server`
+  - room manager, PTY lifecycle, websocket protocol handling
+- `hay/apps/web`
+  - browser terminal client
+- `hay/apps/cli`
+  - local terminal client used by Hop for direct attach flows
+- `mcp/hop-mcp.js`
+  - MCP server exposing Hop terminal/session primitives
+
+## Session Model
+
+- Terminal sessions are identified by stable Hop session names.
+- MCP attaches to daemon-scoped terminal handles, but can recover across daemon
+  restart by reattaching through the stable session identity.
+- Active PTYs live in Hay rooms managed by the host process.
+- Browser users, local CLI attaches, and agents all talk to the same underlying
+  room runtime.
+
+## Build Outputs
+
+- `hay/apps/web/dist/` is the source web build output.
+- `hay-web/` is the synced bundle served by `hop`.
+- `npm run build` from the repo root rebuilds Hay and syncs `hay-web/`.
+
+## Operational Notes
+
+- Hop always uses the external Hay host runtime.
+- `HOP_TERMINAL_BACKEND` is accepted only for compatibility and ignored.
+- Session history and audit logging are per-session and workspace-aware.
+- Alternate-screen/TUI applications are captured with TUI-aware audit behavior.
+
+## Legacy Notes
+
+The earlier `ttyd`/`tmux` approach is now only historical context.
+If you see older references in commit history or archived notes, treat them as
+obsolete unless the current code still uses them explicitly.

package/MULTI-AGENT-CHECKLIST.md

@@ -0,0 +1,37 @@
+# Multi-Agent Run Checklist
+
+## Goal
+Ensure concurrent agent sessions stay isolated, time-bounded, and recoverable.
+
+## Safety Gates
+
+- [ ] **Isolated terminals** — assign each agent its own PTY via `hop_create_terminal`; never share a session between agents.
+- [ ] **Destructive-command approval** — gate every `rm -rf`, force-push, or `DROP TABLE` behind explicit user confirmation, regardless of autonomy level.
+- [ ] **Per-command timeouts** — enforce `idle_ms` and `max_wait_ms` on every `hop_wait_terminal` call; escalate or kill on deadline breach.
+
+## Execution Loop
+
+1. **Create session** — one `hop_create_terminal` per agent.
+2. **Validate prompt** — use `hop_wait_terminal(until_prompt=true, start_from="latest")` before the first command.
+3. **Preferred turn helper** — use `hopx_agent_turn` or `hopx_send_and_wait` for normal multi-turn work.
+4. **Manual fallback** — drop to `hop_write_terminal` + `hop_wait_terminal` only when you need custom wait conditions or lower-level control.
+5. **Assert state** — verify expected output before continuing.
+6. **Repeat** the loop; never write into a terminal that is still busy.
+
+Recommended defaults:
+
+- `capture="readable_raw"`
+- `start_from="latest"` for prompt waits, `start_from="cursor"` for deltas
+- explicit `max_wait_ms`
+- explicit `idle_ms` when using raw wait conditions
+
+## Recovery
+
+| Step | Action |
+|------|--------|
+| 1 | Send `ctrl_c` via `hop_send_key` or `control="interrupt"` via `hopx_agent_turn` |
+| 2 | If still running, continue waiting or poll an async wait job instead of blind-resending |
+| 3 | If unresponsive after deadline, `hop_close_terminal(killSession=true)` |
+| 4 | On orchestrator crash, close all child terminals and verify there are no orphaned sessions |
+
+**Rule:** Never skip escalation. Always tear down — never leave orphaned sessions.

package/README-MCP.md

@@ -0,0 +1,232 @@
+# Hop MCP Server
+
+The Hop MCP server exposes Hop terminal and session APIs over MCP so agents can
+create terminals, stream output, send input, and recover cleanly across daemon
+restarts.
+
+## Quick Start
+
+1. Ensure the Hop daemon is running locally.
+
+2. Auto-configure supported MCP clients:
+
+```bash
+hop-mcp-setup
+```
+
+If you are running from source in this repo:
+
+```bash
+npm run setup-mcp
+```
+
+Supported auto-detected clients:
+- Claude Code
+- Claude Desktop
+- Cursor IDE
+- Gemini CLI
+- Codex CLI
+- VS Code / GitHub Copilot (workspace via `.vscode/mcp.json`)
+- Antigravity
+
+3. Or configure your MCP client manually:
+
+```json
+{
+  "mcpServers": {
+    "hop": {
+      "command": "hop-mcp"
+    }
+  }
+}
+```
+
+4. Restart your MCP client.
+
+The server auto-connects to the local Hop daemon via `~/.hop2/.tunnel-state`.
+
+## Recommended Flow: One Agent Terminal
+
+Recommended flow for driving one Claude/Codex/Gemini terminal safely:
+
+1. Launch a dedicated terminal:
+
+```
+hop_create_terminal(name="subagent", cwd="/path/to/repo")
+```
+
+2. Start the agent CLI:
+
+```
+hop_write_terminal(data="claude\n")
+# or
+hop_write_terminal(data="codex\n")
+```
+
+3. Wait for a prompt before sending work:
+
+```
+hop_wait_terminal(until_prompt=true, start_from="latest")
+```
+
+4. Run a multi-turn loop:
+- Preferred: `hopx_agent_turn(data="<task>", mode="auto")`
+- Good lower-level helper: `hopx_send_and_wait(data="<task>", press_enter=true, capture="readable_raw")`
+- Manual split only when needed:
+- send with `hop_write_terminal(data="<task>\n")`
+- wait with `hop_wait_terminal(capture="readable_raw")`
+- continue via cursor deltas from `hop_read_terminal(start_from="cursor", cursor=...)`
+
+5. Stop or clean up:
+- interrupt with `hop_send_key(key="ctrl_c")`
+- close with `hop_close_terminal(killSession=false)`
+
+Safety notes:
+- Keep one subagent per terminal (no multiplexing multiple agent CLIs in one PTY).
+- For existing user sessions, verify `agentPermitted` (or set with `hop_set_agent_permission`) before writing input.
+
+## Remote Hop
+
+To connect to a remote hop instance, set env vars or use `connect_server`:
+
+```bash
+HOP_API_URL="https://hop2.example.com" HOP_TOKEN="<token>" hop-mcp
+```
+
+Or from tools:
+
+```
+connect_server(base_url="https://hop2.example.com", token="<token>")
+```
+
+Optional verification:
+
+```
+connect_server(base_url="https://hop2.example.com", token="<token>", verify=true, verify_endpoint="/api/sessions")
+```
+
+## Tools
+
+Core tools (`hop_`): stable atomic operations.
+- `connect_server`
+- `hop_server_info`
+- `hop_list_sessions`
+- `hop_list_terminals`
+- `hop_create_terminal`
+- `hop_attach_terminal`
+- `hop_write_terminal`
+- `hop_send_key`
+- `hop_wait_terminal`
+- `hop_wait_start`
+- `hop_wait_poll`
+- `hop_resize_terminal`
+- `hop_read_terminal`
+- `hop_close_terminal`
+- `hop_set_agent_permission`
+- `hop_list_workspaces`
+- `hop_save_workspace`
+- `hop_load_workspace`
+- `hop_use_workspace`
+
+Helper tools (`hopx_`): convenience wrappers built on top of core tools.
+- `hopx_send_and_wait` (single-call send + wait wrapper)
+- `hopx_agent_turn` (single-turn send + wait + mode-aware output, default `mode="auto"`)
+
+## Resources
+
+- `hop://sessions`
+- `hop://terminals`
+- `hop://workspaces`
+
+## Notes
+
+- Agent access is gated by `agentPermitted` for sessions created by users.
+- `terminal_id` is an ephemeral daemon attachment handle; underlying hop `sessionName` is the stable identity.
+- On daemon restart, terminal-scoped tools auto-reattach once using stored session identity and retry transparently when possible.
+- `hop_read_terminal` returns a cursor for incremental polling.
+- Default fallback terminal size is `140x40` when no explicit `cols`/`rows` are provided.
+- `hop_create_terminal` and `hop_attach_terminal` do a short output warmup before returning, reducing first-command races.
+- Hop isolates shell history per session and writes per-session NDJSON audit logs under `~/.hop2/workspaces/<workspace>/logs/<session>/audit.ndjson`.
+- Audit logging automatically switches to diff-suppressed `tui_keyframe` snapshots for alternate-screen/TUI apps while still capturing all input.
+
+### `hop_read_terminal`
+
+- `hop_read_terminal` supports `mode: "ui"` for structured terminal snapshots:
+  - `ui.lines`: visible screen lines
+  - `ui.cursor`: cursor position
+  - `ui.window.strategy`: `cursor_centered` with auto fallback to `densest_nonempty` when needed
+  - `rawTail`: optional lossless tail of recent raw events (for transient output)
+- Hop server now replays buffered terminal events on stream connect, so late attaches get immediate context instead of an empty first read.
+- `hop_read_terminal` supports `mode: "readable_raw"`:
+  - preserves printable text in `events[].text`
+  - includes compact control hints in `events[].controls` (cursor, erase, alt-screen, visibility)
+  - `control_level` tunes control noise:
+  - `full`
+  - `structural` (cursor/edit/screen controls; omits CR/LF noise)
+  - `none` (default; text only)
+  - `noise_filter` tunes status/spinner suppression:
+  - `balanced` (default; suppresses CR/erase rewrite bursts, commits stable lines, strips spinner-prefix/dot animation for dedupe)
+  - in `balanced`, recent shell echo lines for agent-sent input (prompt + command) and prompt-padding artifacts are compacted
+  - `off` (raw parsed text; useful for parser debugging)
+  - in default `none`, empty output frames are dropped to reduce event noise
+  - `coalesce_ms` and `coalesce_max_chars` can merge adjacent output frames for cleaner streaming text
+  - defaults: `coalesce_ms=250`, `coalesce_max_chars=32768`
+  - parses incrementally across chunk boundaries to avoid split ANSI artifacts
+  - reconstructs common inline edits (backspace/cursor rewrites) so text is cleaner for agent parsing
+  - avoids full ANSI noise while retaining interaction structure
+
+### `hop_wait_terminal`
+
+- `hop_send_key` sends normalized named keys (for example `enter`, `esc`, `up`, `ctrl_c`) to improve interactive TUI control without raw escape-string handling.
+- `hop_wait_terminal` blocks on output conditions without client polling loops:
+  - `start_from` controls where matching begins:
+  - `latest` (default): stream tail / new output only
+  - `cursor`: requires `cursor` argument
+  - `beginning`: oldest buffered event
+  - if no explicit condition is provided, it defaults to `until_agent_done=true`
+  - `async=true` starts a background wait job and returns `wait_id` immediately
+  - `until_agent_done` matches when output has started, terminal becomes quiet, and interactive cursor is visible (agent-friendly completion)
+  - regex match via `until_regex`
+  - prompt match via `until_prompt` (optional custom `prompt_regex`)
+  - quiet-period detection via `idle_ms`
+  - bounded event capture in `raw` or `readable_raw`
+  - for `capture="readable_raw"`, supports `control_level`, `noise_filter`, `coalesce_ms`, `coalesce_max_chars`, and optional `includeMetaEvents`
+
+### Async wait helpers
+
+- `hop_wait_start` / `hop_wait_poll` provide explicit async wait orchestration for long-running conditions:
+  - start with `hop_wait_start(...)` and poll with `hop_wait_poll(wait_id=..., wait=true)`
+  - use `consume=true` on poll to remove completed jobs
+
+### `hopx_send_and_wait`
+
+- `hopx_send_and_wait` combines input send + optional enter/key + wait, and defaults to cursor-based delta capture from just before send.
+  - helper defaults are token-thrifty for interactive loops: `capture_max_events=60`, `control_level=none`, `noise_filter=balanced`, `coalesce_ms=350` (unless overridden)
+  - for `capture="readable_raw"`, `text_only` defaults to `true`; this condenses wait payloads by returning joined `wait.text`, setting `wait.events=[]`, `wait.eventCount=0`, and preserving prior count in `wait.originalEventCount` (set `text_only=false` to keep full events)
+- `hop_read_terminal` supports deterministic delta reads:
+  - `start_from`: `beginning` (default), `cursor`, or `latest`
+  - response includes `cursorStart`, `cursorEnd`, and `next_cursor`
+
+### `hopx_agent_turn`
+
+- `hopx_agent_turn` is a helper wrapper (not a core primitive):
+  - default `mode="auto"` chooses `ui` when terminal is in alternate-screen, otherwise `readable_raw`
+  - in `mode="auto"`, if alternate-screen starts during the same turn, it auto-promotes to `ui` for that response
+  - wait-only continuation is supported: omit `data`/`message` and use `control="wait"` to keep waiting on a terminal without sending new input
+  - `async=true` starts the turn wait in the background and returns `wait_id` immediately; continue polling or controlling that turn with `hopx_agent_turn(wait_id=..., wait=true|false, control=...)`
+  - `control="interrupt"` / `control="terminate"` send an explicit interrupt key (default `esc`); `terminate_message` can send a final follow-up after the interrupt
+  - `mode="ui"` returns a UI snapshot payload after wait
+  - for readable modes, `text_only` defaults to `true` and condenses waits the same way as `hopx_send_and_wait` (set `text_only=false` to keep full wait events)
+  - `text_only` is ignored for `mode="ui"` output snapshots
+  - `mode="ui"` defaults `includeRawTail=false` to reduce noisy tail payloads (opt in with `includeRawTail=true`)
+  - `mode="ui"` defaults wait capture to `capture_max_events=0` (override when you need readable wait-event diagnostics)
+  - when using default `until_agent_done` semantics in `mode="ui"`, hopx applies a short UI busy-guard pass to avoid returning while status lines still indicate active work (`esc to interrupt`, `working`, `waiting for process`, etc.)
+  - `mode="readable_raw"` / `mode="raw"` returns the same wait payload shape as `hopx_send_and_wait`
+
+### Error model
+
+- Tool calls that fail at the Hop API layer now return MCP errors (`isError: true`) with normalized fields:
+  - `ok`
+  - `status`
+  - `endpoint`
+  - `error`