susops 3.0.0rc3.dev9__tar.gz → 3.0.0rc5.dev1__tar.gz

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: susops
-Version: 3.0.0rc3.dev9
+Version: 3.0.0rc5.dev1
 Summary: SusOps — SSH SOCKS5 proxy manager with PAC server, Textual TUI, and system tray apps
 License: MIT
 Requires-Python: >=3.11
@@ -315,7 +315,7 @@ susops
 
 ### Screens
 
-**Dashboard** (default) — split-pane view. Left sidebar shows all connections (status dot, SOCKS port, live throughput), PAC server status, and active file shares. Right panel is tabbed:
+**Dashboard** (default) — split-pane view. Left sidebar shows all connections (status dot — `●` green running, `◐` orange pending, `○` dim stopped — SOCKS port, live throughput), PAC server status, and active file shares. Right panel is tabbed:
 - **Stats** — CPU%, memory, active connections, PID for the selected connection
 - **Bandwidth** — live RX and TX line charts (PlotextPlot, 60-sample rolling window, auto-scaled units)
 - **Forwards** — DataTable of all port forwards (direction, local port, local bind, remote port, remote bind, label)
@@ -436,9 +436,11 @@ susops reset [--force]    # Kill all processes, wipe ~/.susops workspace
 |------|---------|
 | `0` | Success / all running |
 | `1` | Error |
-| `2` | Partial (some services stopped) |
+| `2` | Partial (some services stopped, or any connection pending) |
 | `3` | All stopped |
 
+A connection is **pending** when its SSH master is alive but the ControlMaster socket isn't up yet — typically while ssh-agent is waiting on a key unlock, or between reconnect attempts. `start` returns as soon as the master spawns; auth then completes asynchronously up to 60 s later. Slow agent prompts (Vaultwarden, 1Password, hardware keys) no longer freeze the UI.
+
 ---
 
 ## System Tray
@@ -461,7 +463,7 @@ susops-tray
 
 Requires `rumps`: `pip install "susops[tray-mac]"`
 
-The tray icon reflects the current state (running/partial/stopped). State changes arrive over the daemon's SSE `/events` stream. If the stream drops, the listener reconnects within at most 5 seconds. Both tray implementations support the same feature set via native dialogs:
+The tray icon reflects the current state (running / partial-or-pending / stopped). State changes arrive over the daemon's SSE `/events` stream. If the stream drops, the listener reconnects within at most 5 seconds. When TUI + tray are both attached to the same daemon, quitting one keeps the other running — `stop_on_quit` is skipped if any other frontend is still connected. Both tray implementations support the same feature set via native dialogs:
 
 - **Manage** — toggle connection/PAC host/forward enabled state; start, stop, or restart a specific connection
 - **Start / Stop / Restart All** — bulk lifecycle operations across all connections
@@ -552,7 +554,11 @@ Both can be `true` simultaneously — susops will start an SSH slave for TCP and
 
 ### Port assignment
 
-Ports default to `0` (auto-assign). SusOps picks a random free port from the ephemeral range (49152–65535) at start time and saves it back to `config.yaml`. Pass a specific port to `add-connection` or set it in the config to pin it.
+Ports default to `0` (auto-assign). SusOps picks a random free port from the ephemeral range (49152–65535) at start time and saves it back to `config.yaml`. Pass a specific port to `add-connection` or set it in the config to pin it. Ports outside `1–65535` are rejected on `add`.
+
+### Tag format
+
+Connection tags must match `[A-Za-z0-9][A-Za-z0-9._-]{0,63}` (no slashes, no `..`, no leading punctuation). Tags appear in PID-file names, socket paths, log lines, and PAC keys, so the format is intentionally strict.
 
 ### Workspace
 
@@ -860,6 +866,12 @@ susops
 # Run the CLI
 susops ps
 susops ls
+
+# Build local pypi + brew artifacts (mirrors CI, no upload)
+scripts/build-local.sh pypi      # wheel + sdist
+scripts/build-local.sh brew      # SusOps.app + .dmg (macOS only)
+scripts/build-local.sh install-pypi   # install wheel into a throwaway venv
+scripts/build-local.sh install-brew   # copy SusOps.app to /Applications
 ```
 
 ### Project layout
@@ -937,7 +949,7 @@ brew tap mashb1t/susops
 brew install susops
 ```
 
-The formula at `packaging/homebrew/Formula/susops.rb` uses `virtualenv_install_with_resources` to create an isolated Python environment with all dependencies. The cask at `packaging/homebrew/Casks/susops.rb` is for a future `.dmg` distribution of `SusOps.app`.
+The formula at `packaging/homebrew/Formula/susops.rb` uses `virtualenv_install_with_resources` to create an isolated Python environment for the CLI + TUI. The cask at `packaging/homebrew/Casks/susops.rb` installs `SusOps.app` (PyInstaller bundle) for the macOS tray — `brew install --cask susops`.
 
 **Note:** Resource sha256 checksums in the formula must be updated for each release. Generate them with:
 

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/README.md

@@ -291,7 +291,7 @@ susops
 
 ### Screens
 
-**Dashboard** (default) — split-pane view. Left sidebar shows all connections (status dot, SOCKS port, live throughput), PAC server status, and active file shares. Right panel is tabbed:
+**Dashboard** (default) — split-pane view. Left sidebar shows all connections (status dot — `●` green running, `◐` orange pending, `○` dim stopped — SOCKS port, live throughput), PAC server status, and active file shares. Right panel is tabbed:
 - **Stats** — CPU%, memory, active connections, PID for the selected connection
 - **Bandwidth** — live RX and TX line charts (PlotextPlot, 60-sample rolling window, auto-scaled units)
 - **Forwards** — DataTable of all port forwards (direction, local port, local bind, remote port, remote bind, label)
@@ -412,9 +412,11 @@ susops reset [--force]    # Kill all processes, wipe ~/.susops workspace
 |------|---------|
 | `0` | Success / all running |
 | `1` | Error |
-| `2` | Partial (some services stopped) |
+| `2` | Partial (some services stopped, or any connection pending) |
 | `3` | All stopped |
 
+A connection is **pending** when its SSH master is alive but the ControlMaster socket isn't up yet — typically while ssh-agent is waiting on a key unlock, or between reconnect attempts. `start` returns as soon as the master spawns; auth then completes asynchronously up to 60 s later. Slow agent prompts (Vaultwarden, 1Password, hardware keys) no longer freeze the UI.
+
 ---
 
 ## System Tray
@@ -437,7 +439,7 @@ susops-tray
 
 Requires `rumps`: `pip install "susops[tray-mac]"`
 
-The tray icon reflects the current state (running/partial/stopped). State changes arrive over the daemon's SSE `/events` stream. If the stream drops, the listener reconnects within at most 5 seconds. Both tray implementations support the same feature set via native dialogs:
+The tray icon reflects the current state (running / partial-or-pending / stopped). State changes arrive over the daemon's SSE `/events` stream. If the stream drops, the listener reconnects within at most 5 seconds. When TUI + tray are both attached to the same daemon, quitting one keeps the other running — `stop_on_quit` is skipped if any other frontend is still connected. Both tray implementations support the same feature set via native dialogs:
 
 - **Manage** — toggle connection/PAC host/forward enabled state; start, stop, or restart a specific connection
 - **Start / Stop / Restart All** — bulk lifecycle operations across all connections
@@ -528,7 +530,11 @@ Both can be `true` simultaneously — susops will start an SSH slave for TCP and
 
 ### Port assignment
 
-Ports default to `0` (auto-assign). SusOps picks a random free port from the ephemeral range (49152–65535) at start time and saves it back to `config.yaml`. Pass a specific port to `add-connection` or set it in the config to pin it.
+Ports default to `0` (auto-assign). SusOps picks a random free port from the ephemeral range (49152–65535) at start time and saves it back to `config.yaml`. Pass a specific port to `add-connection` or set it in the config to pin it. Ports outside `1–65535` are rejected on `add`.
+
+### Tag format
+
+Connection tags must match `[A-Za-z0-9][A-Za-z0-9._-]{0,63}` (no slashes, no `..`, no leading punctuation). Tags appear in PID-file names, socket paths, log lines, and PAC keys, so the format is intentionally strict.
 
 ### Workspace
 
@@ -836,6 +842,12 @@ susops
 # Run the CLI
 susops ps
 susops ls
+
+# Build local pypi + brew artifacts (mirrors CI, no upload)
+scripts/build-local.sh pypi      # wheel + sdist
+scripts/build-local.sh brew      # SusOps.app + .dmg (macOS only)
+scripts/build-local.sh install-pypi   # install wheel into a throwaway venv
+scripts/build-local.sh install-brew   # copy SusOps.app to /Applications
 ```
 
 ### Project layout
@@ -913,7 +925,7 @@ brew tap mashb1t/susops
 brew install susops
 ```
 
-The formula at `packaging/homebrew/Formula/susops.rb` uses `virtualenv_install_with_resources` to create an isolated Python environment with all dependencies. The cask at `packaging/homebrew/Casks/susops.rb` is for a future `.dmg` distribution of `SusOps.app`.
+The formula at `packaging/homebrew/Formula/susops.rb` uses `virtualenv_install_with_resources` to create an isolated Python environment for the CLI + TUI. The cask at `packaging/homebrew/Casks/susops.rb` installs `SusOps.app` (PyInstaller bundle) for the macOS tray — `brew install --cask susops`.
 
 **Note:** Resource sha256 checksums in the formula must be updated for each release. Generate them with:
 

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "susops"
-version = "3.0.0-rc3.dev9"
+version = "3.0.0-rc5.dev1"
 description = "SusOps — SSH SOCKS5 proxy manager with PAC server, Textual TUI, and system tray apps"
 readme = "README.md"
 license = { text = "MIT" }

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/src/susops/client.py

@@ -13,6 +13,7 @@ from __future__ import annotations
 import os
 import subprocess
 import sys
+import threading
 import time
 import urllib.error
 import urllib.request
@@ -22,7 +23,22 @@ from typing import Any
 from susops.core.rpc_protocol import InvocationRequest, InvocationResponse
 
 _WORKSPACE_DEFAULT = Path.home() / ".susops"
-_DAEMON_SPAWN_TIMEOUT = 5.0
+# 15s rather than 5s gives headroom for slow init paths (ssh agent
+# prompts, network probes) on top of the daemon's "publish port first,
+# restore async" startup ordering. Frontends will still detect a truly
+# dead daemon quickly because they poll the port file every 100ms — the
+# timeout only fires for genuine hangs.
+_DAEMON_SPAWN_TIMEOUT = 15.0
+
+# Backoff to avoid pegging CPU and filling logs when the daemon
+# repeatedly exits during startup (e.g. corrupt config, PAC port
+# squatted). On a fast-fail (daemon exits within FAST_FAIL_WINDOW_S of
+# spawn), the next ensure_daemon_running call within BACKOFF_S sleeps
+# until the window expires before retrying.
+_FAST_FAIL_WINDOW_S = 2.0
+_BACKOFF_S = 5.0
+_last_fast_fail: float = 0.0
+_last_fast_fail_lock = threading.Lock()
 
 
 class DaemonUnavailableError(RuntimeError):
@@ -44,16 +60,38 @@ def _read_port(workspace: Path) -> int | None:
         return None
 
 
+def _pid_is_susops_daemon(pid: int) -> bool:
+    """True only if PID belongs to a live process whose cmdline matches a
+    services_daemon invocation. Defends against PID reuse: SIGKILLing the
+    daemon under load can let an ssh fork (or any other short-lived child)
+    inherit the freed PID, and a bare `os.kill(pid, 0)` liveness probe
+    would then falsely report the daemon as up.
+    """
+    try:
+        import psutil
+    except ImportError:
+        try:
+            os.kill(pid, 0)
+            return True
+        except (OSError, ValueError):
+            return False
+    try:
+        proc = psutil.Process(pid)
+        cmdline = " ".join(proc.cmdline())
+    except (psutil.NoSuchProcess, psutil.AccessDenied):
+        return False
+    return "susops.core.services_daemon" in cmdline or "susops-services" in cmdline
+
+
 def _is_daemon_alive(workspace: Path) -> bool:
     pid_file = _pid_path(workspace)
     if not pid_file.exists():
         return False
     try:
         pid = int(pid_file.read_text().strip())
-        os.kill(pid, 0)  # signal 0 = liveness probe
-        return True
     except (OSError, ValueError):
         return False
+    return _pid_is_susops_daemon(pid)
 
 
 def ensure_daemon_running(workspace: Path = _WORKSPACE_DEFAULT) -> int:
@@ -65,6 +103,7 @@ def ensure_daemon_running(workspace: Path = _WORKSPACE_DEFAULT) -> int:
     (PAC port squatted, peer daemon alive, …) is surfaced to the caller
     instead of getting buried under a generic "didn't come up" message.
     """
+    global _last_fast_fail
     if _is_daemon_alive(workspace):
         port = _read_port(workspace)
         if port:
@@ -88,12 +127,26 @@ def ensure_daemon_running(workspace: Path = _WORKSPACE_DEFAULT) -> int:
                 "` and start again."
             )
 
+    # If we recently fast-failed a spawn (daemon exited within
+    # _FAST_FAIL_WINDOW_S of starting), wait out the backoff window
+    # before respawning. Prevents tight respawn loops when something is
+    # structurally broken (corrupt config, PAC port squatted, etc.).
+    with _last_fast_fail_lock:
+        since_last = time.monotonic() - _last_fast_fail
+    if since_last < _BACKOFF_S:
+        time.sleep(_BACKOFF_S - since_last)
+
+    # DEVNULL (not PIPE) for stderr — the daemon writes its log to
+    # ~/.susops/logs/susops-services.log directly. Capturing as a pipe
+    # would back up once the daemon's log volume exceeded the kernel
+    # buffer (~64 KB on macOS) and freeze every subsequent log call.
+    spawn_start = time.monotonic()
     proc = subprocess.Popen(
         [sys.executable, "-m", "susops.core.services_daemon",
          "--workspace", str(workspace)],
         stdin=subprocess.DEVNULL,
         stdout=subprocess.DEVNULL,
-        stderr=subprocess.PIPE,
+        stderr=subprocess.DEVNULL,
         start_new_session=True,
     )
 
@@ -104,17 +157,29 @@ def ensure_daemon_running(workspace: Path = _WORKSPACE_DEFAULT) -> int:
             if port:
                 return port
         # If the daemon exited (e.g. preflight rejected it), don't keep
-        # polling — surface the reason and bail out immediately.
+        # polling — surface the reason and bail out immediately. The
+        # daemon's log file is the new source of truth for failure
+        # reasons (stderr is no longer piped — see Popen above).
         rc = proc.poll()
         if rc is not None:
+            # Fast fail = structural problem. Record the timestamp so the
+            # next ensure_daemon_running call within _BACKOFF_S sleeps
+            # before retrying.
+            if time.monotonic() - spawn_start < _FAST_FAIL_WINDOW_S:
+                with _last_fast_fail_lock:
+                    _last_fast_fail = time.monotonic()
+            log_path = workspace / "logs" / "susops-services.log"
+            tail = ""
             try:
-                _, err = proc.communicate(timeout=0.5)
-                stderr = err.decode(errors="replace").strip() if err else ""
+                if log_path.exists():
+                    with open(log_path, "r", encoding="utf-8", errors="replace") as f:
+                        lines = f.readlines()[-15:]
+                        tail = "".join(lines).strip()
             except Exception:
-                stderr = ""
+                pass
             msg = (
                     f"Daemon exited during startup (rc={rc})"
-                    + (f":\n{stderr}" if stderr else "")
+                    + (f":\n{tail}" if tail else "")
             )
             raise DaemonUnavailableError(msg)
         time.sleep(0.1)

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/src/susops/core/config.py

@@ -193,39 +193,32 @@ def save_config(config: SusOpsConfig, workspace: Path = WORKSPACE_DEFAULT) -> No
     clicks.
     """
     import os
+    import tempfile
     path = get_config_path(workspace)
     path.parent.mkdir(parents=True, exist_ok=True)
     yaml = YAML()
     yaml.default_flow_style = False
     yaml.indent(mapping=2, sequence=4, offset=2)
-    # Convert to plain dict via model_dump, then save
     data = config.model_dump(mode='python')
-    # Convert enums to their values for serialization
     data['susops_app']['logo_style'] = config.susops_app.logo_style.value
-    # Compute the target mode BEFORE writing: preserve any user-set restrictive
-    # permissions (e.g. chmod 600 on a hardened install) since Path.replace()
-    # would otherwise clobber them with the temp file's umask-derived mode.
-    # New configs default to 0o600 — the file holds share passwords.
     try:
         target_mode = path.stat().st_mode & 0o777
     except OSError:
         target_mode = 0o600
-    # Temp file in the same directory so the rename stays on one filesystem.
-    # Open with O_EXCL so we never overwrite a leftover temp file from a
-    # crashed earlier save (would otherwise inherit its mode).
-    tmp_path = path.with_name(path.name + ".tmp")
+    # Unique tmp file per call so concurrent saves (rapid TUI start/stop runs
+    # each handler in its own executor thread) don't fight over the same name
+    # and crash on tmp.replace(path) after a peer already renamed it away.
+    fd, tmp_name = tempfile.mkstemp(
+        prefix=path.name + ".",
+        suffix=".tmp",
+        dir=str(path.parent),
+    )
+    tmp_path = Path(tmp_name)
     try:
-        fd = os.open(
-            str(tmp_path),
-            os.O_WRONLY | os.O_CREAT | os.O_TRUNC,
-            target_mode,
-        )
         try:
             with os.fdopen(fd, 'w') as f:
                 yaml.dump(data, f)
         except Exception:
-            # fdopen owns fd on success; on failure we may need to close it
-            # if fdopen never took ownership. Best-effort.
             try:
                 os.close(fd)
             except OSError:
@@ -235,7 +228,6 @@ def save_config(config: SusOpsConfig, workspace: Path = WORKSPACE_DEFAULT) -> No
         os.chmod(tmp_path, target_mode)
         tmp_path.replace(path)
     except Exception:
-        # Best-effort cleanup of the temp file if writing failed.
         try:
             tmp_path.unlink(missing_ok=True)
         except Exception:

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/src/susops/core/pac.py

@@ -134,10 +134,20 @@ class PacServer:
     def start(self, port: int, pac_path: Path) -> None:
         """Start the PAC HTTP server on the given port.
 
-        Raises RuntimeError if already running or if port is in use.
+        No-op if already running on the same port (parallel start() callers
+        race here — facade's check-then-act on is_running() can let multiple
+        threads through). Raises RuntimeError if the port is in use by
+        something else.
         """
         if self._server is not None:
-            raise RuntimeError("PAC server is already running")
+            if self._port == port:
+                # Same port + already up: caller's intent is already satisfied.
+                self._pac_path = pac_path
+                return
+            raise RuntimeError(
+                f"PAC server already running on port {self._port}, "
+                f"refusing to also bind {port}"
+            )
 
         self._pac_path = pac_path
 

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/src/susops/core/rpc_server.py

@@ -7,6 +7,7 @@ Anything outside the allowlist below is rejected too — deny-by-default.
 """
 from __future__ import annotations
 
+import asyncio
 import logging
 
 from aiohttp import web
@@ -87,7 +88,15 @@ async def _handle_rpc(request: web.Request) -> web.Response:
         return web.Response(text=resp.to_json(), status=404, content_type="application/json")
 
     try:
-        result = method(*req.args, **req.kwargs)
+        # Method handlers are synchronous — many touch the filesystem or spawn
+        # `ssh -O check` subprocesses (up to 3s each). Running them directly
+        # in the aiohttp event loop blocks every other RPC behind the slow one
+        # and eventually leads to "Daemon RPC failed: timed out". Offload to
+        # the default thread executor so the loop stays responsive.
+        loop = asyncio.get_running_loop()
+        result = await loop.run_in_executor(
+            None, lambda: method(*req.args, **req.kwargs)
+        )
         resp = InvocationResponse(ok=True, result=result)
         return web.Response(text=resp.to_json(), content_type="application/json")
     except Exception as exc:

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/src/susops/core/services_daemon.py

@@ -86,17 +86,21 @@ def _preflight(workspace: Path, log: "logging.Logger") -> None:
         # Won the race uncontested.
         pass
     else:
-        # File exists. Is the holder alive?
+        # File exists. Is the holder alive AND actually our daemon?
+        # PID reuse (an ssh fork inheriting the freed PID after we kill -9
+        # the daemon) would otherwise wedge us in a refusal loop until the
+        # impostor exits. Reuse the same identity check the client does.
         pid_file = _pid_path(workspace)
         try:
             existing_pid = int(pid_file.read_text().strip())
-            os.kill(existing_pid, 0)  # liveness probe
         except (OSError, ValueError):
-            # Stale PID file. Remove + retry the atomic claim once.
+            existing_pid = None
+        from susops.client import _pid_is_susops_daemon
+        if existing_pid is None or not _pid_is_susops_daemon(existing_pid):
+            # Stale (or impostor) PID file. Remove + retry the atomic claim.
             pid_file.unlink(missing_ok=True)
             _port_path(workspace).unlink(missing_ok=True)
             if not _claim_pid_file(workspace):
-                # Some other daemon claimed it between our cleanup and retry.
                 log.error(
                     "another susops-services daemon raced us to start. "
                     "Try again — or run "
@@ -153,9 +157,28 @@ def main() -> int:
     args = parser.parse_args()
     workspace = Path(args.workspace)
 
-    logging.basicConfig(level=logging.INFO,
-                        format="%(asctime)s [services] %(message)s")
+    # Route logging + raw stderr to a workspace-local file. With stderr
+    # piped from the spawner (client.py used stderr=PIPE) the kernel's
+    # 64 KB pipe buffer would fill within minutes of daemon logging,
+    # blocking every subsequent log call — including ones running on
+    # executor threads serving RPC requests. Result: freeze.
+    log_path = workspace / "logs" / "susops-services.log"
+    log_path.parent.mkdir(parents=True, exist_ok=True)
+    logging.basicConfig(
+        level=logging.INFO,
+        format="%(asctime)s [services] %(message)s",
+        handlers=[logging.FileHandler(log_path, mode="a", encoding="utf-8")],
+        force=True,
+    )
     log = logging.getLogger("susops.services")
+    # Anything that bypasses logging (raw `print`, tracebacks) also goes
+    # to the log file rather than the spawner's pipe.
+    try:
+        _stderr_redirect = open(log_path, "a", buffering=1, encoding="utf-8")
+        sys.stderr = _stderr_redirect
+        sys.stdout = _stderr_redirect
+    except Exception:
+        pass
 
     # Install signal handlers BEFORE preflight: SIGTERM arriving in the
     # window between claiming the PID file and entering the try block
@@ -177,7 +200,17 @@ def main() -> int:
         from susops.core.rpc_server import serve
         from susops.facade import SusOpsManager
 
-        mgr = SusOpsManager(workspace=workspace, _enable_background_threads=True)
+        # Defer the SSH/PAC/share probes (_restore_*) until AFTER the RPC
+        # port is bound and the port file is written. Those probes can
+        # block on the ssh agent (e.g. 1Password approval prompt) for
+        # several seconds, and frontends only give up after _DAEMON_SPAWN_TIMEOUT.
+        # By publishing the port first, the daemon is "alive" to frontends
+        # immediately regardless of how slow the agent decides to be.
+        mgr = SusOpsManager(
+            workspace=workspace,
+            _enable_background_threads=True,
+            _skip_restore=True,
+        )
         # CLI `--port` wins. Fall back to the configured `rpc_server_port`,
         # then 0 (auto-allocate). When we auto-allocate, persist the bound
         # port back to config so subsequent spawns reuse it.
@@ -198,6 +231,24 @@ def main() -> int:
         # backoff for several seconds after a fresh daemon spawn.
         sse_port = mgr.ensure_sse_status_server()
 
+        # Now that the daemon is reachable (RPC + SSE), run the deferred
+        # restore work in a background thread so slow ssh-agent prompts
+        # don't keep the daemon process pinned during startup.
+        def _restore_in_background() -> None:
+            try:
+                mgr._restore_pac()
+                if mgr.config.susops_app.restore_shares_on_start:
+                    mgr._restore_shares()
+                mgr._restore_reconnect_monitor()
+            except Exception:
+                log.exception("Background restore failed")
+
+        threading.Thread(
+            target=_restore_in_background,
+            daemon=True,
+            name="susops-restore",
+        ).start()
+
         # Surface daemon-startup details in the in-memory log buffer too so
         # they show up in the TUI Logs tab and the tray Logs window — the
         # `log.info` calls only land on the daemon process's stderr.
@@ -211,13 +262,16 @@ def main() -> int:
             pass
 
         # Idle-shutdown. Exit when the last SSE client disconnects AND there's
-        # no in-flight work (no SSH masters, no shares, no PAC, no watched
-        # connections). Startup gets a small grace so a frontend that calls
-        # ensure_daemon_running() has time to make its first SSE connection
-        # before the periodic check fires.
+        # no in-flight work AND that state holds for _IDLE_CONSECUTIVE_TICKS
+        # consecutive checks. Requiring multiple ticks defends against the
+        # "SSE flap during reconnect → momentary client_count==0 → daemon
+        # exits → frontend respawns it" churn that bit us during heavy
+        # reconnect activity.
         _IDLE_STARTUP_GRACE_S = 3.0
         _IDLE_CHECK_INTERVAL_S = 5.0
+        _IDLE_CONSECUTIVE_TICKS = 3
         startup_time = _time.monotonic()
+        idle_streak = 0  # consecutive ticks where _should_exit() was True
 
         def _should_exit() -> bool:
             if _time.monotonic() - startup_time < _IDLE_STARTUP_GRACE_S:
@@ -229,19 +283,12 @@ def main() -> int:
             return mgr.is_idle()
 
         def _on_clients_changed(count: int) -> None:
-            # Fast path — react immediately when the last SSE client drops.
-            # The periodic check below handles the "no SSE was ever opened"
-            # case (e.g. `susops ps` fires one RPC and exits).
+            # No fast-path exit here anymore — the watcher's consecutive-tick
+            # requirement covers genuine quiescence and rejects flap events
+            # without us having to special-case them.
+            nonlocal idle_streak
             if count > 0:
-                return
-            if _should_exit():
-                msg = "Last client disconnected and no work pending — shutting down"
-                log.info(msg)
-                try:
-                    mgr._log(msg)
-                except Exception:
-                    pass
-                stop_event.set()
+                idle_streak = 0
 
         mgr._status_server.on_clients_changed = _on_clients_changed
         # Surface SSE connect/disconnect in the in-memory log buffer so the
@@ -249,19 +296,26 @@ def main() -> int:
         mgr._status_server.on_log = mgr._log
 
         def _idle_watcher() -> None:
+            nonlocal idle_streak
             while not stop_event.is_set():
                 if stop_event.wait(_IDLE_CHECK_INTERVAL_S):
                     return
                 if _should_exit():
-                    msg = (f"Idle for {_IDLE_CHECK_INTERVAL_S:.0f}s with "
-                           f"no clients — shutting down")
-                    log.info(msg)
-                    try:
-                        mgr._log(msg)
-                    except Exception:
-                        pass
-                    stop_event.set()
-                    return
+                    idle_streak += 1
+                    if idle_streak >= _IDLE_CONSECUTIVE_TICKS:
+                        msg = (
+                            f"Idle for {_IDLE_CHECK_INTERVAL_S * idle_streak:.0f}s "
+                            f"({idle_streak} consecutive checks) — shutting down"
+                        )
+                        log.info(msg)
+                        try:
+                            mgr._log(msg)
+                        except Exception:
+                            pass
+                        stop_event.set()
+                        return
+                else:
+                    idle_streak = 0
 
         threading.Thread(
             target=_idle_watcher, daemon=True, name="susops-idle-watcher",

{susops-3.0.0rc3.dev9 → susops-3.0.0rc5.dev1}/src/susops/core/ssh.py

@@ -8,6 +8,7 @@ Architecture:
 """
 from __future__ import annotations
 
+import os
 import subprocess
 import time
 from pathlib import Path
@@ -249,9 +250,43 @@ def stop_tunnel(
             except (subprocess.TimeoutExpired, FileNotFoundError, OSError):
                 pass
 
+    # Last-resort sweep for orphan masters bound to our socket path.
+    # Under parallel start/stop the PID file can race (start writes PID Y while
+    # stop reads stale PID X, or the reconnect monitor spawns M2 between the
+    # read and the kill), leaving an ssh -N -T -D ... ControlPath=...sock
+    # process alive until its own keepalive timeout fires ~60s later.
+    if workspace is not None:
+        _sweep_orphan_masters(tag, workspace)
+
     return stopped
 
 
+def _sweep_orphan_masters(tag: str, workspace: Path) -> None:
+    """Kill any ssh process whose argv references our ControlPath socket but
+    that ProcessManager isn't tracking. No-op if psutil is unavailable.
+    """
+    try:
+        import psutil
+    except ImportError:
+        return
+    sock_str = str(socket_path(tag, workspace))
+    try:
+        own_pid = os.getpid()
+    except Exception:
+        own_pid = -1
+    for proc in psutil.process_iter(["pid", "name", "cmdline"]):
+        try:
+            if proc.info["pid"] == own_pid:
+                continue
+            if proc.info.get("name") != "ssh":
+                continue
+            cmdline = proc.info.get("cmdline") or []
+            if any(sock_str in arg for arg in cmdline):
+                proc.kill()
+        except (psutil.NoSuchProcess, psutil.AccessDenied):
+            continue
+
+
 def is_tunnel_running(tag: str, process_mgr: ProcessManager) -> bool:
     """Return True if the ControlMaster SSH process for tag is currently running."""
     return process_mgr.is_running(_master_name(tag))
@@ -282,7 +317,12 @@ def find_master_pid(tag: str, workspace: Path) -> int | None:
 def is_socket_alive(tag: str, workspace: Path) -> bool:
     """Return True if the ControlMaster socket is responsive.
 
-    Uses `ssh -O check` to verify the master is healthy.
+    Uses `ssh -O check` to verify the master is healthy. The 1s timeout
+    is tight on purpose: a healthy local control socket replies in <100ms,
+    and anything longer means the master is stuck (e.g. waiting on agent),
+    in which case we should return False quickly so callers move on. The
+    previous 3s timeout could saturate the daemon's executor pool when
+    multiple callers stalled simultaneously.
     """
     sock = socket_path(tag, workspace)
     if not sock.exists():
@@ -291,7 +331,7 @@ def is_socket_alive(tag: str, workspace: Path) -> bool:
         result = subprocess.run(
             ["ssh", "-O", "check", "-o", f"ControlPath={sock}", "placeholder"],
             capture_output=True,
-            timeout=3,
+            timeout=1,
         )
         return result.returncode == 0
     except (subprocess.TimeoutExpired, FileNotFoundError):