hop2 0.9.1 → 0.10.0

package/INTEGRATION.md

@@ -1,92 +1,84 @@
-# Hop + Termshare Integration Plan
+# Hop + Hay Architecture
 
-This branch integrates [termshare](https://github.com/jzthree/termshare) to replace ttyd/tmux.
+This document describes the current Hop terminal architecture.
 
-## Benefits
+The historical `ttyd`/`tmux` backend has been removed from the active runtime.
+Hop now uses Hay, a local terminal-sharing stack built on WebSocket plus
+`node-pty`, with browser, CLI, and MCP clients all operating on the same
+session model.
 
-- **No ttyd/tmux dependencies** - Pure Node.js solution
-- **Built-in presence** - See who's connected, typing indicators
-- **Control modes** - Collaborative or locked single-controller
-- **No injection hacks** - Direct control over web UI
-- **Mobile-ready** - Responsive design with iOS-style keyboard built-in
-- **CLI client** - Connect from real terminals too
+## Current Stack
 
-## Architecture Change
-
-### Before (ttyd)
 ```
-Browser → Hop Auth Proxy → ttyd (per session) → tmux → shell
-                ↓
-        HTML/WebSocket injection for mobile keyboard
+Browser / Hay CLI / Hop MCP
+            |
+            v
+        Hop daemon
+            |
+            v
+     local hay-host process
+            |
+            v
+      Hay room manager
+            |
+            v
+         node-pty
+            |
+            v
+           shell
 ```
 
-### After (termshare)
-```
-Browser → Hop Auth Proxy → Termshare WebSocket → node-pty → shell
-                ↓
-        Native termshare web UI (mobile keyboard built-in)
-```
-
-## Implementation Steps
-
-### 1. Add Dependencies
-- [x] `node-pty-prebuilt-multiarch` - PTY handling
-- [x] `ws` - WebSocket server
-- [x] Copy termshare room/pty logic into hop (`termshare.js`)
-
-### 2. Remove ttyd/tmux Code
-- [x] Remove `spawn('ttyd', [...])` in `getOrCreateSession()`
-- [ ] Remove `HOP_TMUX_CONF`, `HOP_TMUX_WRAPPER` generation (still present but unused)
-- [ ] Remove `listTmuxSessions()`, `waitForTmuxSessionReady()` (still present but unused)
-- [ ] Remove ttyd dependency check (still present but unused)
-- [x] Remove keyboard HTML injection code (now using termshare's built-in)
-- [x] Remove `getIOSKeyboardInjection()` WebSocket patching (still present but unused)
-
-### 3. Add Termshare Integration
-- [x] Port `Room` and `RoomManager` classes from termshare
-- [x] Port `createPty` function
-- [x] Create termshare WebSocket handler at `/ws` endpoint
-- [x] Map hop sessions to termshare rooms
-
-### 4. Update Session Management
-```javascript
-// Before: activeSessions[name] = { ttyd: Process, port: number }
-// After:  activeSessions[name] = { room: Room }
-```
-✅ Completed
-
-### 5. Serve Termshare Web UI
-- [x] Build termshare web client (with mobile keyboard)
-- [x] Serve static files from hop (`termshare-web/` directory)
-- [x] Inject room parameter for session routing
-
-### 6. Keep Hop Features
-- [x] Authentication (TOTP + cookies)
-- [x] Cloudflare tunneling
-- [x] Session URL routing (`/s/<name>/`)
-- [x] Multi-user support
-
-## Key Files Changed
-
-- `hop` - Main server, updated `getOrCreateSession()` and WebSocket handler
-- `termshare.js` - New module with Room, RoomManager, createPty
-- `termshare-web/` - Built termshare web client with mobile keyboard
-
-## Testing
-
-1. Start hop: `node hop`
-2. Open browser to tunnel URL
-3. Verify:
-   - [ ] Login with TOTP works
-   - [ ] Terminal session works
-   - [ ] Multiple clients see same session
-   - [ ] Presence indicators show
-   - [ ] Control modes work
-   - [ ] Mobile UI works without injection
-
-## Migration Notes
-
-- Sessions are not persistent (by design) - shell exits when all clients disconnect
-- tmux scrollback replaced by termshare's output buffer (200KB)
-- iOS keyboard injection replaced by termshare's built-in mobile keyboard
-- The old ttyd/tmux code paths are still present but no longer used; can be removed in a cleanup pass
+Key properties:
+
+- No `ttyd` dependency
+- No `tmux` dependency
+- Shared session model across browser, CLI, and MCP
+- Session-aware audit logging and history isolation
+- Native mobile UI and virtual keyboard support
+- Local host process survives hop daemon restarts
+
+## Main Components
+
+- `hop`
+  - user-facing CLI and daemon
+  - auth, Cloudflare tunnel management, session metadata, workspace handling
+  - serves the built Hay web client from `hay-web/`
+- `scripts/hay-host.js`
+  - persistent local host process for active Hay rooms
+  - isolates PTY/runtime ownership from the daemon lifecycle
+- `hay/apps/server`
+  - room manager, PTY lifecycle, websocket protocol handling
+- `hay/apps/web`
+  - browser terminal client
+- `hay/apps/cli`
+  - local terminal client used by Hop for direct attach flows
+- `mcp/hop-mcp.js`
+  - MCP server exposing Hop terminal/session primitives
+
+## Session Model
+
+- Terminal sessions are identified by stable Hop session names.
+- MCP attaches to daemon-scoped terminal handles, but can recover across daemon
+  restart by reattaching through the stable session identity.
+- Active PTYs live in Hay rooms managed by the host process.
+- Browser users, local CLI attaches, and agents all talk to the same underlying
+  room runtime.
+
+## Build Outputs
+
+- `hay/apps/web/dist/` is the source web build output.
+- `hay-web/` is the synced bundle served by `hop`.
+- `npm run build` from the repo root rebuilds Hay and syncs `hay-web/`.
+
+## Operational Notes
+
+- Hop always uses the external Hay host runtime.
+- `HOP_TERMINAL_BACKEND` is accepted only for compatibility and ignored.
+- Session history and audit logging are per-session and workspace-aware.
+- Alternate-screen/TUI applications are captured with TUI-aware audit behavior.
+
+## Legacy Notes
+
+The earlier `ttyd`/`tmux` approach is now only historical context.
+If you see older references in commit history or archived notes, treat them as
+obsolete unless the current code still uses them explicitly.

package/MULTI-AGENT-CHECKLIST.md

@@ -12,19 +12,26 @@ Ensure concurrent agent sessions stay isolated, time-bounded, and recoverable.
 ## Execution Loop
 
 1. **Create session** — one `hop_create_terminal` per agent.
-2. **Validate prompt** — call `hop_wait_terminal(until_prompt=true)` to confirm the shell is ready before sending any command.
-3. **Send command** — write via `hop_write_terminal`; never blind-write into mid-execution output.
-4. **Await result** — use `until_regex` or `until_prompt` with `max_wait_ms` to capture output and detect completion.
-5. **Assert state** — verify exit code / expected output before proceeding.
-6. **Repeat** steps 2–5 for each command in the plan.
+2. **Validate prompt** — use `hop_wait_terminal(until_prompt=true, start_from="latest")` before the first command.
+3. **Preferred turn helper** — use `hopx_agent_turn` or `hopx_send_and_wait` for normal multi-turn work.
+4. **Manual fallback** — drop to `hop_write_terminal` + `hop_wait_terminal` only when you need custom wait conditions or lower-level control.
+5. **Assert state** — verify expected output before continuing.
+6. **Repeat** the loop; never write into a terminal that is still busy.
+
+Recommended defaults:
+
+- `capture="readable_raw"`
+- `start_from="latest"` for prompt waits, `start_from="cursor"` for deltas
+- explicit `max_wait_ms`
+- explicit `idle_ms` when using raw wait conditions
 
 ## Recovery
 
 | Step | Action |
 |------|--------|
-| 1 | Send `ctrl_c` via `hop_send_key` |
-| 2 | If unresponsive after 5 s → `hop_close_terminal(killSession=true)` |
-| 3 | On orchestrator crash → kill **all** child terminals (kill switch) |
-| 4 | Verify zero orphaned processes before declaring shutdown complete |
+| 1 | Send `ctrl_c` via `hop_send_key` or `control="interrupt"` via `hopx_agent_turn` |
+| 2 | If still running, continue waiting or poll an async wait job instead of blind-resending |
+| 3 | If unresponsive after deadline, `hop_close_terminal(killSession=true)` |
+| 4 | On orchestrator crash, close all child terminals and verify there are no orphaned sessions |
 
 **Rule:** Never skip escalation. Always tear down — never leave orphaned sessions.

package/README-MCP.md

@@ -1,12 +1,14 @@
 # Hop MCP Server
 
-The Hop MCP server exposes hop’s terminal/session APIs over the Model Context Protocol (MCP), so agents can create terminals, stream output, and send input through Hop.
+The Hop MCP server exposes Hop terminal and session APIs over MCP so agents can
+create terminals, stream output, send input, and recover cleanly across daemon
+restarts.
 
 ## Quick Start
 
-1) Ensure Hop daemon is running locally (creates `~/.hop2/.tunnel-state`).
+1. Ensure the Hop daemon is running locally.
 
-2) Auto-configure supported MCP clients:
+2. Auto-configure supported MCP clients:
 
 ```bash
 hop-mcp-setup
@@ -27,7 +29,7 @@ Supported auto-detected clients:
 - VS Code / GitHub Copilot (workspace via `.vscode/mcp.json`)
 - Antigravity
 
-3) Or configure your MCP client manually:
+3. Or configure your MCP client manually:
 
 ```json
 {
@@ -39,21 +41,21 @@ Supported auto-detected clients:
 }
 ```
 
-4) Restart your MCP client.
+4. Restart your MCP client.
 
 The server auto-connects to the local Hop daemon via `~/.hop2/.tunnel-state`.
 
-## One Subagent Terminal (Claude/Codex)
+## Recommended Flow: One Agent Terminal
 
-Recommended flow for driving one agent CLI safely:
+Recommended flow for driving one Claude/Codex/Gemini terminal safely:
 
-1) Launch a dedicated terminal:
+1. Launch a dedicated terminal:
 
 ```
 hop_create_terminal(name="subagent", cwd="/path/to/repo")
 ```
 
-2) Start the agent CLI:
+2. Start the agent CLI:
 
 ```
 hop_write_terminal(data="claude\n")
@@ -61,22 +63,21 @@ hop_write_terminal(data="claude\n")
 hop_write_terminal(data="codex\n")
 ```
 
-3) Wait for a prompt before sending work:
+3. Wait for a prompt before sending work:
 
 ```
 hop_wait_terminal(until_prompt=true, start_from="latest")
 ```
 
-4) Run a multi-turn loop:
-- preferred: `hopx_send_and_wait(data="<task>", press_enter=true, capture="readable_raw")`
-- helper alternative: `hopx_agent_turn(data="<task>", mode="auto")`
-- manual split (if needed):
-- send one instruction with `hop_write_terminal(data="<task>\n")`
-- wait for output with `hop_wait_terminal(capture="readable_raw")` (defaults to `until_agent_done`)
-- continue with cursor-deltas via `hop_read_terminal(start_from="cursor", cursor=...)`
-- send the next instruction only after output is captured
+4. Run a multi-turn loop:
+- Preferred: `hopx_agent_turn(data="<task>", mode="auto")`
+- Good lower-level helper: `hopx_send_and_wait(data="<task>", press_enter=true, capture="readable_raw")`
+- Manual split only when needed:
+- send with `hop_write_terminal(data="<task>\n")`
+- wait with `hop_wait_terminal(capture="readable_raw")`
+- continue via cursor deltas from `hop_read_terminal(start_from="cursor", cursor=...)`
 
-5) Stop/cleanup:
+5. Stop or clean up:
 - interrupt with `hop_send_key(key="ctrl_c")`
 - close with `hop_close_terminal(killSession=false)`
 
@@ -143,6 +144,13 @@ Helper tools (`hopx_`): convenience wrappers built on top of core tools.
 - `terminal_id` is an ephemeral daemon attachment handle; underlying hop `sessionName` is the stable identity.
 - On daemon restart, terminal-scoped tools auto-reattach once using stored session identity and retry transparently when possible.
 - `hop_read_terminal` returns a cursor for incremental polling.
+- Default fallback terminal size is `140x40` when no explicit `cols`/`rows` are provided.
+- `hop_create_terminal` and `hop_attach_terminal` do a short output warmup before returning, reducing first-command races.
+- Hop isolates shell history per session and writes per-session NDJSON audit logs under `~/.hop2/workspaces/<workspace>/logs/<session>/audit.ndjson`.
+- Audit logging automatically switches to diff-suppressed `tui_keyframe` snapshots for alternate-screen/TUI apps while still capturing all input.
+
+### `hop_read_terminal`
+
 - `hop_read_terminal` supports `mode: "ui"` for structured terminal snapshots:
   - `ui.lines`: visible screen lines
   - `ui.cursor`: cursor position
@@ -166,6 +174,9 @@ Helper tools (`hopx_`): convenience wrappers built on top of core tools.
   - parses incrementally across chunk boundaries to avoid split ANSI artifacts
   - reconstructs common inline edits (backspace/cursor rewrites) so text is cleaner for agent parsing
   - avoids full ANSI noise while retaining interaction structure
+
+### `hop_wait_terminal`
+
 - `hop_send_key` sends normalized named keys (for example `enter`, `esc`, `up`, `ctrl_c`) to improve interactive TUI control without raw escape-string handling.
 - `hop_wait_terminal` blocks on output conditions without client polling loops:
   - `start_from` controls where matching begins:
@@ -180,15 +191,24 @@ Helper tools (`hopx_`): convenience wrappers built on top of core tools.
   - quiet-period detection via `idle_ms`
   - bounded event capture in `raw` or `readable_raw`
   - for `capture="readable_raw"`, supports `control_level`, `noise_filter`, `coalesce_ms`, `coalesce_max_chars`, and optional `includeMetaEvents`
+
+### Async wait helpers
+
 - `hop_wait_start` / `hop_wait_poll` provide explicit async wait orchestration for long-running conditions:
   - start with `hop_wait_start(...)` and poll with `hop_wait_poll(wait_id=..., wait=true)`
   - use `consume=true` on poll to remove completed jobs
+
+### `hopx_send_and_wait`
+
 - `hopx_send_and_wait` combines input send + optional enter/key + wait, and defaults to cursor-based delta capture from just before send.
   - helper defaults are token-thrifty for interactive loops: `capture_max_events=60`, `control_level=none`, `noise_filter=balanced`, `coalesce_ms=350` (unless overridden)
   - for `capture="readable_raw"`, `text_only` defaults to `true`; this condenses wait payloads by returning joined `wait.text`, setting `wait.events=[]`, `wait.eventCount=0`, and preserving prior count in `wait.originalEventCount` (set `text_only=false` to keep full events)
 - `hop_read_terminal` supports deterministic delta reads:
   - `start_from`: `beginning` (default), `cursor`, or `latest`
   - response includes `cursorStart`, `cursorEnd`, and `next_cursor`
+
+### `hopx_agent_turn`
+
 - `hopx_agent_turn` is a helper wrapper (not a core primitive):
   - default `mode="auto"` chooses `ui` when terminal is in alternate-screen, otherwise `readable_raw`
   - in `mode="auto"`, if alternate-screen starts during the same turn, it auto-promotes to `ui` for that response
@@ -202,12 +222,11 @@ Helper tools (`hopx_`): convenience wrappers built on top of core tools.
   - `mode="ui"` defaults wait capture to `capture_max_events=0` (override when you need readable wait-event diagnostics)
   - when using default `until_agent_done` semantics in `mode="ui"`, hopx applies a short UI busy-guard pass to avoid returning while status lines still indicate active work (`esc to interrupt`, `working`, `waiting for process`, etc.)
   - `mode="readable_raw"` / `mode="raw"` returns the same wait payload shape as `hopx_send_and_wait`
+
+### Error model
+
 - Tool calls that fail at the Hop API layer now return MCP errors (`isError: true`) with normalized fields:
   - `ok`
   - `status`
   - `endpoint`
   - `error`
-- Hop now isolates shell history per session and writes per-session NDJSON audit logs under `~/.hop2/workspaces/<workspace>/logs/<session>/audit.ndjson` with truncation metadata for large chunks.
-- Audit logging auto-switches to diff-suppressed `tui_keyframe` snapshots when alternate-screen apps (vim/htop/agent TUIs) are detected, while still capturing all input events.
-- Default fallback terminal size is `140x40` when no explicit `cols`/`rows` are provided.
-- `hop_create_terminal` / `hop_attach_terminal` now do a short output warmup before returning, reducing first-command races in interactive shells.