golem-vm-provider 0.1.54__py3-none-any.whl → 0.1.56__py3-none-any.whl

{golem_vm_provider-0.1.54.dist-info → golem_vm_provider-0.1.56.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: golem-vm-provider
-Version: 0.1.54
+Version: 0.1.56
 Summary: VM on Golem Provider Node - Run your own provider node to offer VMs on the Golem Network
 Keywords: golem,vm,provider,cloud,decentralized
 Author: Phillip Jensen
@@ -61,6 +61,69 @@ pip install golem-vm-provider
 golem-provider start --network testnet
 ```
 
+Verify your environment and connectivity anytime:
+
+```bash
+golem-provider status
+```
+This checks Multipass availability, local/external port reachability, and whether an update is available on PyPI.
+
+### Status Command (TTY and JSON)
+
+Use `golem-provider status` to quickly assess health.
+
+TTY output highlights
+
+```
+Overall         Error | Issues detected | Healthy
+
+Multipass       ✅ OK | ❌ Missing
+
+Provider Port   0.0.0.0:7466
+  Local         ✅ service is listening | ❌ port unavailable
+  External      ✅ reachable | ❌ unreachable — <reason>
+
+SSH Ports       <start>-<end> — OK | limited — N issue(s) | blocked
+  Usable free   <count>   # free AND externally reachable
+  In use        <count>
+  Issues        e.g. "100 not reachable externally" or "3 unreachable, 1 not listening"
+```
+
+Severity rules
+
+- Overall is Error when any critical prerequisite fails:
+  - Provider API port not externally reachable (or external check fails).
+  - No externally reachable SSH ports in the configured range.
+  - Multipass missing or provider local port not ready.
+- Otherwise it shows Issues detected or Healthy.
+
+Machine‑readable JSON
+
+```bash
+golem-provider status --json
+```
+
+Key fields:
+
+- `overall.status`: "healthy" | "issues" | "error"
+- `overall.issues`: list of concise issue strings
+- `ports.provider`:
+  - `port`: int, `host`: string
+  - `status`: "reachable" | "unreachable" (external check failures are treated as "unreachable")
+- `ports.ssh`:
+  - `range`: [start, end)
+  - `status`: "ok" | "limited" | "blocked"
+  - `usable_free`: integer — free AND externally reachable
+  - `in_use`: integer
+  - `issues`: `{ unreachable: int, not_listening: int }`
+  - `ports`: array of per‑port summaries:
+    - `{ port: int, status: "reachable" | "unreachable" | "unknown", listening: bool }`
+
+Notes
+
+- The concept of "free" in JSON is replaced by `usable_free` (free + externally reachable) to avoid misleading counts when ports are blocked.
+- When the external checker is unavailable, per‑port `status` is `"unknown"` and `listening` still reflects local state.
+
 3) Set pricing in USD (GLM rates auto‑compute):
 
 ```bash
@@ -510,6 +573,7 @@ sequenceDiagram
     participant DS as Discovery Service
 
     P->>RT: Initialize
+    P->>RT: Sync with existing VMs
     RT->>AD: Register Callback
     loop Every 4 minutes
         AD->>RT: Get Resources
@@ -519,6 +583,8 @@ sequenceDiagram
     end
 ```
 
+On startup, the provider syncs the resource tracker with all VMs currently running on the host (via Multipass). This ensures advertisements reflect already-allocated CPU, RAM, and storage after restarts, preventing false “outdated advertisement” updates when existing VMs are consuming resources. The sync is based on actual VMs present, independent of any still-open payment streams.
+
 ### Monitoring
 
 The provider includes comprehensive logging:
@@ -589,13 +655,16 @@ The provider includes real-time port verification status:
 
 Example status output:
 
-```bash
-🌟 Port Verification Status
-==========================
-[✅] Provider Port {provider_port}: External ✓ | Internal ✓
-[✅] VM Access Ports: 3 ports available ({start_port}-{start_port+2})
-[✅] Overall Status: Provider Ready
-└─ Can handle up to {n} concurrent VMs
+```
+Overall         Healthy
+
+Provider Port   {host}:{provider_port}
+  Local         ✅ service is listening
+  External      ✅ reachable
+
+SSH Ports       {start_port}-{end_port_minus_one} — OK
+  Usable free   {usable_free}
+  In use        {in_use}
 ```
 
 ### Resource Allocation Issues

{golem_vm_provider-0.1.54.dist-info → golem_vm_provider-0.1.56.dist-info}/RECORD

@@ -13,15 +13,15 @@ provider/discovery/multi_advertiser.py,sha256=_J79wA1-XQ4GsLzt9KrKpWigGSGBqtut7D
 provider/discovery/resource_monitor.py,sha256=AmiEc7yBGEGXCunQ-QKmVgosDX3gOhK1Y58LJZXrwAs,949
 provider/discovery/resource_tracker.py,sha256=MP7IXd3aIMsjB4xz5Oj9zFDTEnvrnw-Cyxpl33xcJcc,6006
 provider/discovery/service.py,sha256=vX_mVSxvn3arnb2cKDM_SeJp1ZgPdImP2aUubeXgdRg,915
-provider/main.py,sha256=_j92g56B-d8CE09Ugv0fqWVMi5jw_iuTrysxSw7845A,32309
-provider/network/port_verifier.py,sha256=3l6WNwBHydggJRFYkAsuBp1eCxaU619kjWuM-zSVj2o,13267
+provider/main.py,sha256=ozyvQc-1oNLznwr09gYdbGBkYncw-1ejdOdNhxOPcTo,52788
+provider/network/port_verifier.py,sha256=mlSzr9Z-W5Z5mL3EYg4zemgGoi8Z5ebNoeFgLGRaoH4,13253
 provider/payments/blockchain_service.py,sha256=4GrzDKwCSUVoENqjD4RLyJ0qwBOJKMyVk5Li-XNsyTc,3567
-provider/payments/monitor.py,sha256=Rw17zYsxZre0zU6R0oeRNvVIzMdXLsgoUvSPHpJy6I0,4488
+provider/payments/monitor.py,sha256=seo8vE622IdbcRE3x69IpvHn2mel_tlMNGt_DxOIoww,5386
 provider/payments/stream_map.py,sha256=qk6Y8hS72DplAifZ0ZMWPHBAyc_3IWIQyWUBuCU3_To,1191
 provider/security/ethereum.py,sha256=EwPZj4JR8OEpto6LhKjuuT3Z9pBX6P7-UQaqJtqFkYQ,1242
 provider/security/faucet.py,sha256=8T4lW1fVQgUk8EQILgbrr9UUosw9e7eA40tlZ2_KCPQ,4368
 provider/security/l2_faucet.py,sha256=yRV4xdPBgU8-LDTLqtuAijfgIoe2kYxvXqJLxFd-BVI,2662
-provider/service.py,sha256=IIjeSM9T4r616nBRnxCUum_sgbyRusMMcja3yQd8zQI,3383
+provider/service.py,sha256=hlQn0woppsYFHZDMEgq-40cOjmiPWruiWLy_dQvaCRU,6859
 provider/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 provider/utils/ascii_art.py,sha256=ykBFsztk57GIiz1NJ-EII5UvN74iECqQL4h9VmiW6Z8,3161
 provider/utils/logging.py,sha256=VV3oTYSRT8hUejtXLuua1M6kCHmIJgPspIkzsUVhYW0,1920
@@ -39,7 +39,7 @@ provider/vm/port_manager.py,sha256=iYSwjTjD_ziOhG8aI7juKHw1OwwRUTJQyQoRUNQvz9w,1
 provider/vm/provider.py,sha256=A7QN89EJjcSS40_SmKeinG1Jp_NGffJaLse-XdKciAs,1164
 provider/vm/proxy_manager.py,sha256=n4NTsyz2rtrvjtf_ceKBk-g2q_mzqPwruB1q7UlQVBc,14928
 provider/vm/service.py,sha256=Ki4SGNIZUq3XmaPMwAOoNzdZzKQsmFXid374wgjFPes,4636
-golem_vm_provider-0.1.54.dist-info/METADATA,sha256=u7WDC-uHV2F5NJ1ayjL7Vlz2s1yXDRFrOIolk_7V1EM,18452
-golem_vm_provider-0.1.54.dist-info/WHEEL,sha256=b4K_helf-jlQoXBBETfwnf4B04YC67LOev0jo4fX5m8,88
-golem_vm_provider-0.1.54.dist-info/entry_points.txt,sha256=5Jiie1dIXygmxmDW66bKKxQpmBLJ7leSKRrb8bkQALw,52
-golem_vm_provider-0.1.54.dist-info/RECORD,,
+golem_vm_provider-0.1.56.dist-info/METADATA,sha256=D83fVjgeQlLJN2XzW_XGGBWK0g0PAe0nOp87hzjgLjU,20932
+golem_vm_provider-0.1.56.dist-info/WHEEL,sha256=b4K_helf-jlQoXBBETfwnf4B04YC67LOev0jo4fX5m8,88
+golem_vm_provider-0.1.56.dist-info/entry_points.txt,sha256=5Jiie1dIXygmxmDW66bKKxQpmBLJ7leSKRrb8bkQALw,52
+golem_vm_provider-0.1.56.dist-info/RECORD,,

provider/main.py

@@ -143,6 +143,540 @@ def main():
     return
 
 
+def _get_installed_version(pkg_name: str) -> str:
+    try:
+        return metadata.version(pkg_name)
+    except Exception:
+        return "unknown"
+
+
+def _get_latest_version_from_pypi(pkg_name: str) -> Optional[str]:
+    # Avoid network in pytest runs
+    if os.environ.get("PYTEST_CURRENT_TEST"):
+        return None
+    try:
+        import json as _json
+        from urllib.request import urlopen
+        with urlopen(f"https://pypi.org/pypi/{pkg_name}/json", timeout=5) as resp:
+            data = _json.loads(resp.read().decode("utf-8"))
+            return data.get("info", {}).get("version")
+    except Exception:
+        return None
+
+
+@cli.command("status")
+def status(json_out: bool = typer.Option(False, "--json", help="Output machine-readable JSON")):
+    """Show provider environment status and update info (pretty or JSON)."""
+    from .utils.logging import logger as _logger
+    from rich.console import Console
+    from rich.table import Table
+    from rich.panel import Panel
+    from rich import box
+
+    # Temporarily quiet info logs during checks for cleaner UI
+    prev_level = _logger.level
+    try:
+        _logger.setLevel("WARNING")
+    except Exception:
+        pass
+
+    # Silence port_verifier warnings during status checks for clean UI
+    import logging as _logging
+    try:
+        _pv_logger = _logging.getLogger("provider.network.port_verifier")
+        _prev_pv_level = _pv_logger.level
+        _pv_logger.setLevel(_logging.CRITICAL)
+    except Exception:
+        _pv_logger = None
+        _prev_pv_level = None
+    # Also quiet config auto-detection logs (e.g., multipass path) for clean JSON/TTY
+    try:
+        _cfg_logger = _logging.getLogger("provider.config")
+        _prev_cfg_level = _cfg_logger.level
+        _cfg_logger.setLevel(_logging.WARNING)
+    except Exception:
+        _cfg_logger = None
+        _prev_cfg_level = None
+
+    # Defer config-heavy imports until after log levels are adjusted
+    from .config import settings as _settings
+    from .network.port_verifier import PortVerifier
+
+    # Versions
+    pkg = "golem-vm-provider"
+    current = _get_installed_version(pkg)
+    latest = _get_latest_version_from_pypi(pkg)
+    update_available = bool(latest and current != latest)
+
+    # Environment
+    env = os.environ.get("GOLEM_PROVIDER_ENVIRONMENT", _settings.ENVIRONMENT)
+    net = getattr(_settings, "NETWORK", None)
+    dev_mode = env == "development" or bool(getattr(_settings, "DEV_MODE", False))
+
+    # Multipass
+    mp = {"ok": False, "path": None, "version": None, "error": None}
+    try:
+        mp_path = _settings.MULTIPASS_BINARY_PATH
+        mp["path"] = mp_path or None
+        if mp_path:
+            import subprocess
+            r = subprocess.run([mp_path, "version"], capture_output=True, text=True, timeout=5)
+            if r.returncode == 0:
+                mp["ok"] = True
+                mp["version"] = (r.stdout or r.stderr).strip()
+            else:
+                mp["error"] = (r.stderr or r.stdout or "failed").strip()
+        else:
+            mp["error"] = "not configured"
+    except Exception as e:
+        mp["ok"] = False
+        mp["error"] = str(e)
+
+    # Provider port (local)
+    port = int(_settings.PORT)
+    host = getattr(_settings, "HOST", "0.0.0.0")
+    local = {"ok": False, "detail": None}
+    try:
+        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        s.settimeout(1)
+        local_conn = s.connect_ex(("127.0.0.1", port)) == 0
+        s.close()
+        if local_conn:
+            local["ok"] = True
+            local["detail"] = "service is listening"
+        else:
+            # Check that we can bind (port free)
+            if asyncio.run(verify_provider_port(port)):
+                local["ok"] = True
+                local["detail"] = "port is free (bindable)"
+            else:
+                local["ok"] = False
+                local["detail"] = "port unavailable"
+    except Exception as e:
+        local["ok"] = False
+        local["detail"] = str(e)
+
+    # Always use shared external port-checker for public reachability
+    servers = ["http://195.201.39.101:9000"]
+
+    external = {"status": "unknown", "verified_by": None, "error": None}
+    try:
+        verifier = PortVerifier(servers, discovery_port=port)
+        results = asyncio.run(verifier.verify_external_access({port}))
+        r = results.get(port)
+        if r and r.accessible:
+            external["status"] = "reachable"
+            external["verified_by"] = r.verified_by
+        elif r:
+            external["status"] = "unreachable"
+            external["error"] = r.error
+        else:
+            external["status"] = "not_verified"
+    except Exception as e:
+        external["status"] = "check_failed"
+        external["error"] = str(e).splitlines()[0]
+
+    # Base data structure
+    data = {
+        "version": {
+            "installed": current,
+            "latest": latest,
+            "update_available": update_available,
+        },
+        "environment": {
+            "environment": env,
+            "network": net,
+            "dev_mode": dev_mode,
+        },
+        "multipass": mp,
+        "ports": {
+            "provider": {
+                "port": port,
+                "host": host,
+                "local_ok": local["ok"],
+                "local_detail": local["detail"],
+                "external": external,
+            }
+        },
+    }
+
+    # SSH port usage summary from state file + external reachability for full range
+    try:
+        from pathlib import Path as _Path
+        import json as _json
+        state_path = _Path(_settings.PROXY_STATE_DIR) / "ports.json"
+        ports_in_use = []
+        if state_path.exists():
+            with open(state_path, "r") as fh:
+                st = _json.load(fh)
+            for _req_name, pinfo in (st.get("proxies", {}) or {}).items():
+                prt = pinfo.get("port")
+                if isinstance(prt, int):
+                    ports_in_use.append(prt)
+        start = int(getattr(_settings, "PORT_RANGE_START", 50800))
+        end = int(getattr(_settings, "PORT_RANGE_END", 50900))
+        total = max(0, end - start)
+        used = sorted(set(prt for prt in ports_in_use if start <= prt < end))
+        # Check if used ports are actually listening
+        used_listening = []
+        used_not_listening = []
+        for prt in used:
+            try:
+                s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+                s.settimeout(0.5)
+                ok = s.connect_ex(("127.0.0.1", prt)) == 0
+                s.close()
+                (used_listening if ok else used_not_listening).append(prt)
+            except Exception:
+                used_not_listening.append(prt)
+        free_count = total - len(used)
+        # External reachability across entire range
+        external_ok: set[int] = set()
+        ext_results_map: dict[int, bool] = {}
+        external_batch_ok = False
+        external_batch_error: str | None = None
+        try:
+            verifier_all = PortVerifier(servers, discovery_port=port)
+            _ext_results = asyncio.run(verifier_all.verify_external_access(set(range(start, end))))
+            for prt, res in _ext_results.items():
+                p = int(prt)
+                ok = bool(getattr(res, "accessible", False))
+                ext_results_map[p] = ok
+                if ok:
+                    external_ok.add(p)
+            external_batch_ok = True
+        except Exception:
+            # Leave external_ok empty on failure
+            external_batch_ok = False
+            try:
+                external_batch_error = str(_)  # type: ignore[name-defined]
+            except Exception:
+                external_batch_error = None
+
+        firewall_issues = [p for p in used_listening if p not in external_ok]
+
+        # Build per-port details for JSON consumers
+        details = []
+        for p in range(start, end):
+            details.append({
+                "port": p,
+                "in_use": p in used,
+                "local_listening": p in used_listening,
+                "external_reachable": bool(ext_results_map.get(p, False)) if external_batch_ok else False,
+            })
+
+        # Compute number of free ports that are actually externally reachable (usable)
+        usable_free_count = None
+        try:
+            if external_batch_ok:
+                usable_free_count = len([
+                    p for p in range(start, end)
+                    if (p not in used) and bool(ext_results_map.get(p, False))
+                ])
+        except Exception:
+            usable_free_count = None
+
+        # Legacy detailed metrics (retained under ssh_legacy)
+        _ssh_legacy = {
+            "range": [start, end],
+            "total": total,
+            "in_use": used,
+            "listening_ok": used_listening,
+            "listening_issues": used_not_listening,
+            "free_count": free_count,
+            "external_reachable_count": len([p for p in range(start, end) if p in external_ok]),
+            "firewall_issues_count": len(firewall_issues),
+            "external_checked": external_batch_ok,
+            "external_error": external_batch_error,
+            "usable_free_count": usable_free_count,
+            "details": details,
+        }
+
+        # Concise status fields for programmatic checks (mirrors TTY)
+        _ext_reach = int(len(external_ok))
+        _issues_count = len(used_not_listening) + len(firewall_issues)
+        if (not external_batch_ok) or _ext_reach == 0:
+            ssh_status = "blocked"
+        elif _issues_count > 0:
+            ssh_status = "limited"
+        else:
+            ssh_status = "ok"
+
+        # Usable free: default to 0 when external check failed
+        if usable_free_count is None:
+            usable_free_out = 0
+        else:
+            usable_free_out = int(usable_free_count)
+
+        # Issues breakdown matching TTY wording
+        unreachable_count = total if (not external_batch_ok or _ext_reach == 0) else len(firewall_issues)
+        not_listening_count = len(used_not_listening)
+
+        # Build minimal per-port status list for JSON consumers
+        # Consistent definition:
+        # - status: reachable | unreachable | unknown (unknown only if external check failed)
+        # - listening: true | false
+        ports_detail: list[dict] = []
+        for p in range(start, end):
+            listening = p in used_listening
+            if external_batch_ok:
+                status_val = "reachable" if bool(ext_results_map.get(p, False)) else "unreachable"
+            else:
+                status_val = "unknown"
+            ports_detail.append({
+                "port": p,
+                "status": status_val,
+                "listening": bool(listening),
+            })
+
+        data["ports"]["ssh"] = {
+            "range": [start, end],
+            "status": ssh_status,
+            "usable_free": usable_free_out,
+            "in_use": len(used),
+            "issues": {
+                "unreachable": int(unreachable_count),
+                "not_listening": int(not_listening_count),
+            },
+            "ports": ports_detail,
+        }
+
+    except Exception:
+        # Non-fatal; omit ssh summary if state not available
+        pass
+
+    # Provider concise status: reachable | unreachable (treat check failures as unreachable)
+    prov_status = external.get("status")
+    provider_status = "reachable" if prov_status == "reachable" else "unreachable"
+    data["ports"]["provider"]["status"] = provider_status
+
+    # Compute overall and issues for JSON output (mirrors condensed model)
+    json_issues = []
+    json_ssh_blocked = False
+    json_critical_no_ssh = False
+    # Multipass
+    if not mp["ok"]:
+        json_issues.append("Multipass not available")
+    # Provider local port
+    if not local["ok"]:
+        json_issues.append(f"Provider port {port} not ready")
+    # SSH ports
+    if data["ports"].get("ssh"):
+        _ssh = data["ports"]["ssh"]
+        _status = str(_ssh.get("status") or "blocked").lower()
+        _issues = _ssh.get("issues") or {}
+        _not_listening = int(_issues.get("not_listening", 0) or 0)
+        _unreachable = int(_issues.get("unreachable", 0) or 0)
+        if _status == "blocked":
+            json_ssh_blocked = True
+            json_critical_no_ssh = True
+            json_issues.append("No externally reachable SSH ports")
+        else:
+            if _unreachable > 0:
+                json_issues.append(f"{_unreachable} SSH port(s) unreachable externally")
+        if _not_listening > 0:
+            json_issues.append(f"{_not_listening} SSH port(s) not listening")
+    # Provider external
+    json_critical_provider_external = False
+    if external.get("status") in ("unreachable", "check_failed"):
+        json_issues.append("Provider API port not reachable externally")
+        json_critical_provider_external = True
+
+    if json_critical_no_ssh or (not local["ok"]) or (not mp["ok"]) or json_critical_provider_external:
+        overall_status = "error"
+    else:
+        overall_status = "healthy" if (not json_issues and not json_ssh_blocked) else "issues"
+
+    data["overall"] = {"status": overall_status, "issues": json_issues}
+
+    if json_out:
+        import json as _json
+        print(_json.dumps(data, indent=2))
+        # Restore logger level
+        try:
+            _logger.setLevel(prev_level)
+        except Exception:
+            pass
+        if _pv_logger and _prev_pv_level is not None:
+            try:
+                _pv_logger.setLevel(_prev_pv_level)
+            except Exception:
+                pass
+        if _cfg_logger and _prev_cfg_level is not None:
+            try:
+                _cfg_logger.setLevel(_prev_cfg_level)
+            except Exception:
+                pass
+        return
+
+    console = Console()
+
+    # Overall status
+    issues = []
+    if not mp["ok"]:
+        issues.append("Multipass not available")
+    if not local["ok"]:
+        issues.append(f"Provider port {port} not ready")
+    ssh_blocked = False
+    critical_no_ssh = False
+    if data["ports"].get("ssh"):
+        ssh = data["ports"]["ssh"]
+        if ssh.get("listening_issues"):
+            issues.append(f"{len(ssh['listening_issues'])} SSH port(s) not listening")
+        if ssh.get("free_count", 0) == 0:
+            issues.append("No free SSH ports available")
+        if ssh.get("external_checked"):
+            if int(ssh.get("external_reachable_count", 0) or 0) == 0:
+                ssh_blocked = True
+                critical_no_ssh = True  # No externally reachable SSH ports is critical
+                issues.append("No externally reachable SSH ports")
+        else:
+            # If we couldn't check, mark as issue but not critical
+            issues.append("SSH external reachability check failed")
+    critical_provider_external = False
+    if external["status"] in ("unreachable", "check_failed"):
+        issues.append("Provider API port not reachable externally")
+        critical_provider_external = True
+
+    # Severity: Error when critical conditions are met; else Issues/Healthy
+    if critical_no_ssh or (not local["ok"]) or (not mp["ok"]) or critical_provider_external:
+        overall = "Error"
+    else:
+        overall = "Healthy" if (not issues and not ssh_blocked) else "Issues detected"
+
+    # Build a single compact table
+    tbl = Table(box=box.SIMPLE_HEAVY, show_header=False, pad_edge=False)
+    tbl.add_column("Item", style="bold")
+    tbl.add_column("Value")
+
+    # Header
+    if overall == "Healthy":
+        overall_txt = "[green]Healthy[/green]"
+    elif overall == "Error":
+        overall_txt = "[red]Error[/red]"
+    else:
+        overall_txt = f"[yellow]{overall}[/yellow]"
+    tbl.add_row("Overall", overall_txt)
+    tbl.add_row("", "")
+
+    # Versions
+    tbl.add_row("Versions", "")
+    ver_inst = data["version"]["installed"] or "unknown"
+    ver_latest = data["version"]["latest"] or "unknown"
+    upd = data["version"]["update_available"]
+    tbl.add_row("  Installed", f"[white]{ver_inst}[/white]")
+    if upd:
+        tbl.add_row("  Latest", f"[bold bright_yellow]{ver_latest}[/bold bright_yellow]  [grey62](pip install -U golem-vm-provider)[/grey62]")
+        tbl.add_row("  Update", "[bold bright_yellow]⬆️  yes[/bold bright_yellow]")
+    else:
+        tbl.add_row("  Latest", f"[cyan]{ver_latest}[/cyan]")
+        tbl.add_row("  Update", "[green]no[/green]")
+    tbl.add_row("", "")
+
+    # Environment
+    tbl.add_row("Environment", "")
+    tbl.add_row("  Environment", env + (" (dev)" if dev_mode else ""))
+    tbl.add_row("  Network", net or "-")
+    tbl.add_row("", "")
+
+    # Multipass
+    mp_ver = (mp.get("version") or mp.get("error") or "-").replace("\n", ", ")
+    tbl.add_row("Multipass", "")
+    tbl.add_row("  Status", "✅ OK" if mp["ok"] else "❌ Missing")
+    tbl.add_row("  Path", mp.get("path") or "-")
+    tbl.add_row("  Version", mp_ver)
+    tbl.add_row("", "")
+
+    # Provider port
+    tbl.add_row("Provider Port", f"{host}:{port}")
+    tbl.add_row("  Local", ("✅ " if local["ok"] else "❌ ") + (local["detail"] or ""))
+    # External reachability is foundational; treat unreachable and check failures the same
+    _ext = external.get("status") or "unknown"
+    _err = external.get("error")
+    if _ext == "reachable":
+        ext_row = "✅ reachable"
+    elif _ext in ("unreachable", "check_failed"):
+        ext_row = "❌ unreachable" + (f" — {_err}" if _err else "")
+    elif _ext == "not_verified":
+        ext_row = "⚠️ not verified"
+    else:
+        ext_row = "⚠️ " + _ext + (f" — {_err}" if _err else "")
+    tbl.add_row("  External", ext_row)
+
+    # SSH ports (condensed, actionable)
+    if data["ports"].get("ssh"):
+        ssh = data["ports"]["ssh"]
+        r0, r1 = ssh['range'][0], ssh['range'][1]-1
+        tbl.add_row("", "")
+        status_val = str(ssh.get("status") or "blocked").lower()
+        issues_obj = ssh.get("issues") or {}
+        unreachable_issues = int(issues_obj.get("unreachable", 0) or 0)
+        not_listening_issues = int(issues_obj.get("not_listening", 0) or 0)
+        in_use = int(ssh.get("in_use", 0) or 0)
+        usable_free = ssh.get("usable_free")
+
+        # Determine clear status
+        if status_val == "ok":
+            status_txt = "[green]OK[/green]"
+        elif status_val == "limited":
+            status_txt = f"[yellow]limited — {not_listening_issues + (unreachable_issues or 0)} issue(s)[/yellow]"
+        else:
+            status_txt = "[red]blocked[/red]"
+
+        tbl.add_row("SSH Ports", f"{r0}-{r1} — {status_txt}")
+
+        # Provide only the most relevant numbers
+        # Usable free = free and externally reachable; avoid misleading "Free" when blocked
+        tbl.add_row("  Usable free", str(int(usable_free or 0)))
+        tbl.add_row("  In use", str(in_use))
+        if status_val == "blocked":
+            # Show total not reachable externally
+            total_ports = (r1 - r0 + 1)
+            cnt = unreachable_issues if unreachable_issues else total_ports
+            tbl.add_row("  Issues", f"{cnt} not reachable externally")
+        elif (not_listening_issues or unreachable_issues):
+            parts = []
+            if unreachable_issues:
+                parts.append(f"{unreachable_issues} unreachable (listening but blocked)")
+            if not_listening_issues:
+                parts.append(f"{not_listening_issues} not listening")
+            tbl.add_row("  Issues", ", ".join(parts))
+
+    # Issues / Tips combined at bottom
+    # Only show Notes when there are issues
+    if issues:
+        tbl.add_row("", "")
+        tbl.add_row("Issues", "\n".join(f"• {t}" for t in issues))
+
+    console.print(Panel(tbl, title="Provider Status"))
+
+    # Tips
+    tips = []
+    if update_available:
+        tips.append("Upgrade with: pip install -U golem-vm-provider")
+    if not mp["ok"]:
+        tips.append("Install Multipass and/or set GOLEM_PROVIDER_MULTIPASS_BINARY_PATH")
+    if external["status"] != "reachable":
+        tips.append("Ensure at least one port-check server is online (see above)")
+    # Tips are included in the single panel under Notes
+
+    # Restore logger level
+    try:
+        _logger.setLevel(prev_level)
+    except Exception:
+        pass
+    if _pv_logger and _prev_pv_level is not None:
+        try:
+            _pv_logger.setLevel(_prev_pv_level)
+        except Exception:
+            pass
+    if _cfg_logger and _prev_cfg_level is not None:
+        try:
+            _cfg_logger.setLevel(_prev_cfg_level)
+        except Exception:
+            pass
+
+
 @wallet_app.command("faucet-l2")
 def wallet_faucet_l2():
     """Request L2 faucet funds for the provider's payment address (native ETH)."""

provider/network/port_verifier.py

@@ -126,8 +126,8 @@ class PortVerifier:
         Returns:
             Dictionary mapping ports to their verification results
         """
-        results = {}
-        attempts = []
+        results: Dict[int, PortVerificationResult] = {}
+        attempts: List[ServerAttempt] = []
         
         # Try each server
         for server in self.port_check_servers:
@@ -146,27 +146,29 @@ class PortVerifier:
                     
                     if response.status == 200:
                         data = await response.json()
-                        if data["success"]:
-                            # Convert server results to PortVerificationResult objects
-                            for port_str, result in data["results"].items():
-                                port = int(port_str)
-                                if port not in results or not results[port].accessible:
-                                    # Only update if we haven't found a successful verification yet
-                                    results[port] = PortVerificationResult(
-                                        port=port,
-                                        accessible=result["accessible"],
-                                        error=result.get("error"),
-                                        verified_by=server if result["accessible"] else None,
-                                        attempts=[]  # Will be filled at the end
-                                    )
-                            attempts.append(ServerAttempt(server=server, success=True))
-                            logger.info(f"Port verification completed using {server}")
-                        else:
-                            attempts.append(ServerAttempt(
-                                server=server,
-                                success=False,
-                                error=f"Server {server} returned unsuccessful response"
-                            ))
+                        # Treat a 200 response as a successful attempt regardless of overall success flag.
+                        # The 'success' field in the checker indicates if any port was reachable, not server health.
+                        raw_results = data.get("results", {}) or {}
+                        for port_key, result in raw_results.items():
+                            try:
+                                port = int(port_key)
+                            except Exception:
+                                # Some implementations might already use ints
+                                port = int(result.get("port", 0)) if isinstance(result, dict) else 0
+                            if not port:
+                                continue
+                            accessible = bool(result.get("accessible"))
+                            err = result.get("error")
+                            if port not in results or (accessible and not results[port].accessible):
+                                results[port] = PortVerificationResult(
+                                    port=port,
+                                    accessible=accessible,
+                                    error=err,
+                                    verified_by=server if accessible else None,
+                                    attempts=[],
+                                )
+                        attempts.append(ServerAttempt(server=server, success=True))
+                        logger.info(f"Port verification completed using {server}")
                     else:
                         attempts.append(ServerAttempt(
                             server=server,
@@ -198,7 +200,7 @@ class PortVerifier:
                 ))
                 logger.warning(error_msg)
         
-        # If no servers were successful, fail verification
+        # If no servers responded successfully, fail verification
         if not any(attempt.success for attempt in attempts):
             error_msg = (
                 "Failed to connect to any port check servers. Please ensure:\n"
@@ -209,19 +211,15 @@ class PortVerifier:
             logger.error(error_msg)
             raise RuntimeError(error_msg)
 
-        # If no successful verifications but servers were reachable, mark ports as inaccessible
-        if not any(result.accessible for result in results.values()):
-            error_msg = "No ports were verified as accessible"
-            logger.error(error_msg)
-            results = {
-                port: PortVerificationResult(
+        # Ensure all requested ports are present in results; default to inaccessible
+        for port in ports:
+            if port not in results:
+                results[port] = PortVerificationResult(
                     port=port,
                     accessible=False,
-                    error=error_msg,
-                    attempts=[]  # Will be filled below
+                    error=None,
+                    attempts=[],
                 )
-                for port in ports
-            }
         
         # Add attempts to all results
         for result in results.values():

provider/payments/monitor.py

@@ -56,21 +56,41 @@ class StreamMonitor:
                         logger.warning(f"stream {stream_id} lookup failed: {e}")
                         continue
                     # Stop VM if remaining runway < threshold
-                    remaining = max(s["stopTime"] - now, 0)
+                    remaining = max(int(s["stopTime"]) - int(now), 0)
                     logger.debug(
                         f"stream {stream_id} for VM {vm_id}: start={s['startTime']} stop={s['stopTime']} "
                         f"rate={s['ratePerSecond']} withdrawn={s['withdrawn']} halted={s['halted']} remaining={remaining}s"
                     )
-                    if self._get("STREAM_MONITOR_ENABLED", False) and remaining < int(self._get("STREAM_MIN_REMAINING_SECONDS", 0)):
-                        logger.info(f"Stopping VM {vm_id} due to low stream runway ({remaining}s)")
+                    # If stream is force-halted, delete immediately to free all resources
+                    if bool(s.get("halted")):
+                        logger.info(
+                            f"Deleting VM {vm_id} due to halted stream (id={stream_id}, now={now})"
+                        )
+                        try:
+                            await self.vm_service.delete_vm(vm_id)
+                        except Exception as e:
+                            logger.warning(f"delete_vm failed for {vm_id}: {e}")
+                        try:
+                            await self.stream_map.remove(vm_id)
+                        except Exception as e:
+                            logger.debug(f"failed to remove vm {vm_id} from stream map: {e}")
+                        continue
+
+                    # Only stop a VM when runway is completely empty
+                    if remaining == 0:
+                        logger.info(
+                            f"Stopping VM {vm_id} as stream runway is exhausted (id={stream_id}, now={now}, stop={s.get('stopTime')})"
+                        )
                         try:
                             await self.vm_service.stop_vm(vm_id)
                         except Exception as e:
                             logger.warning(f"stop_vm failed for {vm_id}: {e}")
-                    else:
-                        logger.debug(
-                            f"VM {vm_id} stream {stream_id} healthy (remaining={remaining}s, threshold={self._get('STREAM_MIN_REMAINING_SECONDS', 0)}s)"
-                        )
+                        continue
+
+                    # Otherwise, do not stop; just log health and consider withdrawals
+                    logger.debug(
+                        f"VM {vm_id} stream {stream_id} healthy (remaining={remaining}s)"
+                    )
                     # Withdraw if enough vested and configured
                     if self._get("STREAM_WITHDRAW_ENABLED", False) and self.client:
                         vested = max(min(now, s["stopTime"]) - s["startTime"], 0) * s["ratePerSecond"]

provider/service.py

@@ -37,6 +37,68 @@ class ProviderService:
             # Initialize services
             await self.port_manager.initialize()
             await self.vm_service.provider.initialize()
+
+            # Before starting advertisement, sync allocated resources with existing VMs
+            try:
+                vm_resources = await self.vm_service.get_all_vms_resources()
+                await self.vm_service.resource_tracker.sync_with_multipass(vm_resources)
+            except Exception as e:
+                logger.warning(f"Failed to sync resources with existing VMs: {e}")
+
+            # Cross-check running VMs against payment streams. If a VM has no
+            # active stream, it is no longer rented: terminate it and free resources.
+            try:
+                # Only perform checks if payments are configured
+                if settings.STREAM_PAYMENT_ADDRESS and not settings.STREAM_PAYMENT_ADDRESS.lower().endswith("0000000000000000000000000000000000000000") and settings.POLYGON_RPC_URL:
+                    stream_map = app.container.stream_map()
+                    reader = app.container.stream_reader()
+
+                    # Use the most recent view of VMs from the previous sync
+                    vm_ids = list(vm_resources.keys()) if 'vm_resources' in locals() else []
+                    for vm_id in vm_ids:
+                        try:
+                            stream_id = await stream_map.get(vm_id)
+                        except Exception:
+                            stream_id = None
+
+                        if stream_id is None:
+                            reason = "no stream mapped"
+                            should_terminate = True
+                        else:
+                            try:
+                                ok, msg = reader.verify_stream(int(stream_id), settings.PROVIDER_ID)
+                                should_terminate = not ok
+                                reason = msg if not ok else "ok"
+                            except Exception as e:
+                                # If verification cannot be performed, be conservative and keep the VM
+                                logger.warning(f"Stream verification error for VM {vm_id} (stream {stream_id}): {e}")
+                                should_terminate = False
+                                reason = f"verification error: {e}"
+
+                        if should_terminate:
+                            logger.info(
+                                f"Deleting VM {vm_id}: inactive stream (stream_id={stream_id}, reason={reason})"
+                            )
+                            try:
+                                await self.vm_service.delete_vm(vm_id)
+                            except Exception as e:
+                                logger.warning(f"Failed to delete VM {vm_id}: {e}")
+                            try:
+                                await stream_map.remove(vm_id)
+                            except Exception:
+                                pass
+
+                    # Re-sync after any terminations to ensure ads reflect capacity
+                    try:
+                        vm_resources = await self.vm_service.get_all_vms_resources()
+                        await self.vm_service.resource_tracker.sync_with_multipass(vm_resources)
+                    except Exception as e:
+                        logger.warning(f"Post-termination resource sync failed: {e}")
+                else:
+                    logger.info("Payments not configured; skipping startup stream checks")
+            except Exception as e:
+                logger.warning(f"Failed to reconcile VMs with payment streams: {e}")
+
             await self.advertisement_service.start()
             # Start pricing auto-updater; trigger re-advertise after updates
             async def _on_price_updated(platform: str, glm_usd):