@neoline/hostbridge 2.0.4 → 2.0.5

package/README.md

@@ -18,13 +18,13 @@ This repo is portable: it ships with no personal server paths or built-in hosts.
 - `session_read(session_id, timeout=1.0, max_chars=20000)` — read available raw output after interactive writes.
 - `file_write_text(session_id, remote_path, content)` — write a UTF-8 text file through the active shell, up to 100MiB.
 - `file_read_text(session_id, remote_path)` — read a UTF-8 text file through the active shell, up to 100MiB.
-- `file_upload(session_id, local_path, remote_path)` — upload a local file with chunked shell/base64 transfer, up to 100MiB.
-- `file_download(session_id, remote_path, local_path)` — download a remote file with chunked shell/base64 transfer, up to 100MiB.
+- `file_upload(session_id, local_path, remote_path)` — stream a local file as binary data, up to 100MiB.
+- `file_download(session_id, remote_path, local_path)` — stream a remote file as binary data, up to 100MiB.
 - `session_close(session_id)` — close a persistent session.
 
 Every tool returns a structured payload with a stable lowercase `error_code`, such as `not_found`, `denied`, `remote_error`, `timeout`, or `internal`.
 
-## Install and configure v1
+## Install and configure
 
 Install from npm or Python, then create a schema-v1 configuration and start the daemon:
 
@@ -47,14 +47,14 @@ hostbridge hosts check gpu
 
 ## File transfer model
 
-HostBridge file transfer is shell-based because recorded login paths may be JumpServer menus, OTP flows, wrapper scripts, or `docker exec` sessions rather than standard SSH/SFTP connections. The first-class transfer tools stream chunked base64 through the already-open shell session, verify hashes, and work for text or binary artifacts up to 100MiB:
+HostBridge streams file data as raw bytes over its bounded multiplexed tunnel. Direct SSH targets use SFTP on the host's shared AsyncSSH connection. JumpServer, menu, wrapper, and `docker exec` paths use the persistent PTY remote agent, which implements the same binary upload/download protocol without Base64 expansion. Both providers verify SHA-256 and size before atomically replacing the destination. Transfers are limited to 100MiB by default:
 
 ```bash
 file_upload(session_id, "/absolute/local/script.py", "/remote/work/script.py")
 file_download(session_id, "/remote/work/result.log", "/absolute/local/result.log")
 ```
 
-Give file tools local and remote paths; do not paste base64 into tool arguments. Files larger than 100MiB are deliberately rejected with guidance to use remote pull instead, for example `curl`, `wget`, `git clone`, `modelscope download`, or another remote-native downloader. The MCP file tools intentionally keep this shell/base64 backend for non-standard login flows. The optional `mock-ssh` bridge below exposes SFTP/SCP protocol compatibility on top of the same backend for clients which require SSH file-transfer channels.
+Give file tools local and remote paths. Files larger than 100MiB are deliberately rejected with guidance to use remote pull instead, for example `curl`, `wget`, `git clone`, `modelscope download`, or another remote-native downloader. The optional `mock-ssh` bridge below exposes SFTP/SCP protocol compatibility on top of the same binary transfer service for clients which require SSH file-transfer channels.
 
 ## Mock SSH bridge
 
@@ -90,9 +90,9 @@ The bridge supports:
 - OpenSSH `scp` in its default SFTP-backed mode.
 - Legacy OpenSSH `scp -O` mode through AsyncSSH's SCP server.
 
-Each SSH channel opens its own HostBridge backend session and closes it when the channel ends, so an interactive shell `exit` does not kill later exec or file-transfer requests. The SFTP/SCP implementation is a compatibility layer backed by HostBridge's shell/base64 transfer primitives, so it keeps working across JumpServer/menu/wrapper login flows where direct remote SFTP is unavailable. It inherits the same transfer size limit (100MiB by default); for larger files, start a remote-native pull from the target host.
+Each SSH channel owns a HostBridge runtime lease and closes it when the channel ends, while all leases for a host reuse the daemon's single provider connection. An interactive shell `exit` therefore does not kill later exec or file-transfer requests. SFTP/SCP uses the same binary transfer service and works across JumpServer/menu/wrapper login flows where direct remote SFTP is unavailable. It inherits the same transfer size limit (100MiB by default); for larger files, start a remote-native pull from the target host.
 
-Mock SSH exec channels are byte-oriented. Binary stdin is staged with one total deadline, client EOF is propagated to the remote command, and disconnect, timeout, cancellation, or a missing backend exit status closes the HostBridge session. PTY command output is tail-bounded before base64 encoding, so arbitrary bytes are preserved without accumulating unbounded local output.
+Mock SSH exec channels are byte-oriented. Binary stdin is streamed with one total deadline, client EOF is propagated to the remote command, and disconnect, timeout, cancellation, or a missing backend exit status closes the HostBridge session. Output remains bytes end to end and is bounded before returning to the client.
 
 ## Daemon lifecycle
 
@@ -104,13 +104,13 @@ hostbridge daemon status
 hostbridge daemon stop
 ```
 
-The daemon uses a Unix domain socket with a small JSON-line RPC protocol by default, not HTTP and not a TCP port:
+The daemon uses a Unix domain socket with a versioned binary multiplexing protocol, not HTTP and not a TCP port:
 
 ```text
 ~/.hostbridge/daemon.sock
 ```
 
-Every client validates both protocol version and the exact HostBridge runtime version. When an older or incompatible daemon owns the configured socket, the launcher shuts it down and starts the current runtime before use. The daemon accepts at most 64 concurrent control connections; an idle control connection is closed after 30 seconds so abandoned clients cannot consume unbounded worker threads.
+Every client validates both protocol version and the exact HostBridge runtime version. When an older or incompatible daemon owns the configured socket, the launcher shuts it down and starts the current runtime before use. Each control connection has bounded pending requests, concurrent handlers, receive windows, and per-stream flow-control credit. Business handlers execute on the daemon event loop; there is no synchronous service worker pool.
 
 When launched through the npm wrapper on macOS, HostBridge sets `HOSTBRIDGE_DAEMON_SOCKET` to `/private/tmp/hostbridge-$USER/daemon.sock` so sandboxed MCP launchers do not accidentally isolate the daemon under a rewritten `HOME`. Python-only launches keep the `~/.hostbridge/daemon.sock` default unless you set `HOSTBRIDGE_DAEMON_SOCKET` or `HOSTBRIDGE_DAEMON_DIR`.
 
@@ -132,6 +132,8 @@ For MCP clients, prefer setting the environment variable instead of relying on s
 
 Sessions can outlive a single agent client process. HostBridge records an optional `HOSTBRIDGE_AGENT_ID` owner on sessions and refuses cross-owner operations unless an internal cleanup path force-closes them. Configure `limits.max_sessions` per host to cap concurrent live sessions (default `16`).
 
+See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for runtime ownership, cancellation, and provider invariants.
+
 ## Configuration lifecycle
 
 `hostbridge setup` creates an empty schema-v1 file with mode `0600`. It does not run an interactive login recorder. Use `hostbridge hosts add` for new entries and `hostbridge migrate OLD NEW` for legacy files. The daemon loads one validated config at startup; restart it after configuration changes.

package/docs/ARCHITECTURE.md

@@ -0,0 +1,62 @@
+# HostBridge Runtime Architecture
+
+HostBridge has one asynchronous runtime owned by the local daemon. CLI, MCP, and Mock SSH clients use the same versioned binary Mux protocol over a private Unix socket.
+
+## Ownership
+
+```text
+CLI / MCP / Mock SSH
+        |
+        v
+MuxDaemonServer
+        |
+        +-- AsyncHostBridgeServices -- RuntimeServiceSession -- HostRuntimeLease
+        |
+        +-- forwarding stream registry ------------------------+
+                                                               |
+                                                               v
+                                                     HostTunnelManager
+                                                               |
+                                                    one HostRuntime per host
+                                                       /              \
+                                             Native AsyncSSH       PTY agent
+```
+
+`HostTunnelManager` owns exactly one active provider generation per host. Session leases and forwarding stream handles share that provider. A Native SSH provider owns one AsyncSSH connection; a PTY provider owns one local login process and one persistent remote Mux agent.
+
+## Data Plane
+
+- Exec stdin, stdout, and stderr are bytes.
+- Upload, download, forwarding, and shell traffic use binary Mux streams.
+- Connection and stream credit provide bounded memory and backpressure.
+- Uploads write a private temporary file, validate expected size and SHA-256, fsync, chmod, and atomically replace the destination.
+- No host data path uses JSONL, Base64, shell line framing, or periodic polling.
+
+## Session Lifecycle
+
+```text
+OPENING -> ACTIVE -> DRAINING -> CLOSED
+```
+
+Opening reserves host session capacity before acquiring a runtime lease. Cancellation or rejection after acquisition closes the lease before releasing the reservation.
+
+Closing marks the session as draining before releasing the registry lock. Normal close rejects active operations. Force close cancels and bounds active operation shutdown, terminates registered remote jobs, deletes their job directories, closes the lease-owned shell, releases the lease, and removes all session metadata. Finalization runs in its own shielded task, so caller cancellation cannot interrupt resource cleanup.
+
+Idle expiry uses one asyncio task on the daemon event loop. Service audit events cover session, command, transfer, and remote-task lifecycle transitions.
+
+## Provider Isolation
+
+- Shell identity is the internal lease token, never a client-provided shell ID.
+- Shell creation and removal are atomic per provider, preventing duplicate processes during concurrent first access.
+- Cancelling a Native SSH exec channel does not abort the shared host connection.
+- PTY exec and shell processes run in dedicated process groups and use TERM/KILL cleanup deadlines.
+- Provider generation changes invalidate old leases and stream handles without attaching them to a replacement connection.
+
+## Shutdown Order
+
+1. Stop accepting Mux work and drain or cancel request handlers.
+2. Force-close service sessions and their operations, jobs, and shells.
+3. Close forwarding streams and host providers.
+4. Remove daemon socket and PID files.
+
+There is no synchronous service implementation or daemon worker executor. `HostBridgeClient` remains a thread-safe synchronous facade for current CLI, MCP, and Mock SSH callers, but it communicates with the same asynchronous daemon runtime and does not create per-host connections.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@neoline/hostbridge",
-  "version": "2.0.4",
+  "version": "2.0.5",
   "description": "HostBridge MCP server for persistent allowlisted SSH and shell sessions.",
   "type": "module",
   "license": "MIT",
@@ -25,6 +25,7 @@
     "src/server_control_mcp/**/*.py",
     "pyproject.toml",
     "README.md",
+    "docs/ARCHITECTURE.md",
     "LICENSE",
     "examples/"
   ],

package/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "hostbridge"
-version = "2.0.4"
+version = "2.0.5"
 description = "HostBridge MCP server for persistent allowlisted SSH and shell sessions."
 readme = "README.md"
 requires-python = ">=3.11"

package/src/server_control_mcp/__init__.py

@@ -3,13 +3,13 @@
 from __future__ import annotations
 
 import os
-import subprocess
+import subprocess  # nosec B404
 import sys
 from importlib import metadata
 from typing import NoReturn
 
 __all__ = ["__version__", "ensure_runtime"]
-__version__ = "2.0.4"
+__version__ = "2.0.5"
 
 # (import_name, requirement) pairs for runtime dependency probing.
 _REQUIRED_DEPENDENCIES = (
@@ -102,7 +102,8 @@ def _auto_install_enabled() -> bool:
 
 def _install_dependencies(requirements: list[str]) -> bool:
     print(f"hostbridge: installing Python dependencies: {', '.join(requirements)}", file=sys.stderr)
-    result = subprocess.run(
+    # The executable and pip arguments are package-owned constants.
+    result = subprocess.run(  # nosec B603
         [sys.executable, "-m", "pip", "install", *requirements], stdout=sys.stderr, stderr=sys.stderr, check=False
     )
     return result.returncode == 0

package/src/server_control_mcp/async_lifecycle.py

@@ -0,0 +1,41 @@
+"""Shared lifecycle helpers for paired asynchronous pumps."""
+
+from __future__ import annotations
+
+import asyncio
+from typing import TypeVar
+
+T = TypeVar("T")
+
+
+async def finish_task_despite_cancellation(task: asyncio.Future[T]) -> T:
+    """Wait for cleanup without allowing repeated caller cancellation to detach it."""
+    while True:
+        try:
+            return await asyncio.shield(task)
+        except asyncio.CancelledError:
+            if task.done():
+                return task.result()
+
+
+async def finish_task_before_cancellation(task: asyncio.Future[T]) -> T:
+    """Finish an owned task before propagating cancellation from its waiter."""
+    try:
+        return await asyncio.shield(task)
+    except asyncio.CancelledError:
+        await finish_task_despite_cancellation(task)
+        raise
+
+
+async def stop_input_after_peer_eof(task: asyncio.Task[None]) -> None:
+    """Stop pending input after peer EOF while preserving completed failures."""
+    if task.done():
+        await task
+        return
+    task.cancel()
+    results = await asyncio.shield(asyncio.gather(task, return_exceptions=True))
+    result = results[0]
+    if isinstance(result, asyncio.CancelledError):
+        return
+    if isinstance(result, BaseException):
+        raise result

package/src/server_control_mcp/cli.py

@@ -78,7 +78,7 @@ def build_parser() -> argparse.ArgumentParser:
     check.add_argument("host_id")
     check.add_argument("--timeout", type=int, default=30)
 
-    daemon = commands.add_parser("daemon", help="manage the protocol-v1 daemon")
+    daemon = commands.add_parser("daemon", help="manage the binary Mux daemon")
     _add_config_argument(daemon)
     daemon_commands = daemon.add_subparsers(dest="daemon_command", required=True)
     start = daemon_commands.add_parser("start")