dsh-pve-vm 0.1.1 → 0.2.0

package/CHANGELOG.md

@@ -3,6 +3,35 @@
 All notable changes to this plugin are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), versions follow [SemVer](https://semver.org/).
 
+## [0.2.0] — 2026-10-13
+
+Watch the agent work: a live VM screen in the right sidebar of the web UI.
+
+### Added
+
+* **The “VM screen” tab** — a right-sidebar panel that streams the frames the agent sees. The frames
+  come from the agent's own RFB session, so no second VNC session appears and the `qm vncproxy`
+  password is never reset; captures and tools share one session queue. Watch-only by design: the
+  panel injects nothing into the guest.
+* **`viewAutoOpen`** (off by default) — reveal the tab by itself when the agent touches a console.
+* **`viewFollowAgent`** (on by default) — the panel follows the VM the agent works with; a VM chip
+  pins it by hand. Watching never counts as agent work: frame requests suppress the activity mark
+  (`withConsole(..., { mark: false })`), so the panel cannot chase itself.
+* **`viewFps`** (2) and **`viewScale`** (2) — frame pace and resolution divisor, both changeable on
+  the fly inside the panel.
+* **Host routes** `/pve-vm/console/state`, `/frame` and `/release` on the harness web server,
+  registered only when the composition has a `webServer`. They carry a loopback-only fence: Host must
+  be loopback, a present Origin must match Host, `sec-fetch-site: cross-site` is refused and POST must
+  be `application/json`. A harness bound to a LAN therefore keeps the panel dark.
+* `RfbClient.waitForUpdate({ idleOk: true })` — silence at a message boundary means "the screen did
+  not change" instead of an error, while a timeout inside a message stays fatal.
+
+### Fixed
+
+* **`inputChannel` and `sendkeyDelayMs` never reached the tools.** Both keys were missing from
+  `CONFIG_KEYS`, so `snapshot()` dropped them and every run silently used the built-in `auto`/80 ms
+  defaults. The form accepted the values, the code ignored them.
+
 ## [0.1.1] — 2026-10-13
 
 Linux guest support. Everything below was found by running the plugin against a real RedOS 8

package/README.md

@@ -30,6 +30,9 @@ agent ──► plugin ──► ssh ──► node: qm vncproxy ──► unix
 * **Clipboard** — `console_paste` writes UTF-8 text into the guest clipboard over RFB and presses
   Ctrl+V; `console_clipboard` reads the guest clipboard back, so text can leave the VM without OCR.
 * **A settings card** in the web UI (English and Russian labels), backed by the profile patch layer.
+* **A live screen panel** — the “VM screen” tab in the right sidebar shows the frames the agent
+  sees, taken from the very same VNC session, while the agent works. It follows the agent's VM
+  automatically or stays on the VM you pick by hand. Watching only: the panel never clicks or types.
 * **Zero runtime dependencies** — RFB, DES authentication, keysym tables and PNG encoding are
   implemented in this package on top of `node:crypto` and `node:zlib`. There is nothing to install
   on the host, and nothing to install on the node either.
@@ -95,6 +98,10 @@ layer, and the same values can be edited there by hand:
     sessionIdleMs: 300000
     guestTimeoutMs: 60000
     guestTypeCommand: ''        # script for typing non-ASCII text into a Linux guest
+    viewAutoOpen: false         # open the "VM screen" tab by itself when the agent touches a console
+    viewFollowAgent: true       # the panel follows the VM the agent works with
+    viewFps: 2                  # frame pace in the panel, 1–4
+    viewScale: 2                # frame resolution divisor: 1 native, 2 half, 4 quarter
 ```
 
 **First run checklist**
@@ -137,6 +144,33 @@ Every action tool accepts a common tail: `observe` (return a frame after the act
 | `vm_type_text` | `vmid`, `text`, `settleMs?`, `observe?` | Type text through the best available channel: ASCII → console, non-ASCII → the user session on Windows / `xdotool`–`wtype` on Linux. |
 | `vm_status` | `vmid?` | Is the VM allowed, does the agent answer, which OS, are there open console sessions. |
 
+## Watch the VM screen live
+
+The **VM screen** tab in the right sidebar shows what the agent sees while it works. Add it from the
+sidebar guide, or let `viewAutoOpen` reveal it the moment the agent touches a console.
+
+* The frames are taken from the agent's own RFB session — **no second VNC session appears** and the
+  `qm vncproxy` password is never reset under the agent's feet. Captures and the agent's tools share
+  one session queue, so they never tear the stream for each other.
+* `viewFollowAgent` keeps the panel on the VM the agent is working with; clicking a VM chip pins the
+  panel to that VM instead. `viewFps` (1–4) and `viewScale` (1 / 2 / 4) control the pace and the
+  resolution, and both can be changed on the fly inside the panel.
+* The panel is **watch-only**: no clicks, no typing, nothing is injected into the guest. A blank
+  frame is called out explicitly — the guest blanked its display or is locked.
+* Frames are served by the plugin's own routes (`/pve-vm/console/frame`, `/state`, `/release`) on the
+  harness web server, behind a loopback-only fence: a request is answered only when the `Host` is
+  loopback (`localhost`, `127.0.0.0/8`, `[::1]`), a present `Origin` matches `Host`, and
+  `sec-fetch-site: cross-site` is refused. If you bind the harness to your LAN, the panel stays dark
+  by design — the same fence the harness applies to `/api`.
+* **The fence is loopback, not the harness token.** These routes live outside `/api`, so the
+  harness URL token does not cover them (the browser-skill overview works the same way). Any
+  process on the same machine can therefore pull frames or call `/release`; that is the accepted
+  price of loading frames without a token on a single-user box. Serve VMs you would not mind a
+  local user seeing, and harden with a harness-issued token if that is not acceptable.
+* The console session stays open while the panel streams (the idle reaper does not take it); the
+  panel's “disconnect console” button closes it immediately. Streaming pauses by itself when the tab
+  is hidden.
+
 ## How it works
 
 * **Transport.** A VM has no TCP VNC port: QEMU listens on the unix socket

package/client.js

@@ -77,6 +77,43 @@ window.__ModuleLoader__.load({
         'field.guestTypeCommand': 'Ввод не-ASCII в Linux-госте',
         'field.guestTypeCommandHint':
           'Скрипт ввода не-ASCII текста в Linux-госте; получает base64 текста первым аргументом.',
+        'field.viewAutoOpen': 'Открывать панель экрана сама',
+        'field.viewAutoOpenHint':
+          'Когда агент прикасается к консоли ВМ, вкладка «Экран ВМ» раскрывается сама в правом сайдбаре.',
+        'field.viewFollowAgent': 'Панель следует за агентом',
+        'field.viewFollowAgentHint':
+          'Показывать ВМ, с которой работает агент. Выключено — панель держит ВМ, выбранную вручную.',
+        'field.viewFps': 'Кадров в секунду в панели',
+        'field.viewFpsHint':
+          '1–4. Два кадра в секунду — рабочий темп: движение агента видно, узел не перегружен.',
+        'field.viewScale': 'Делитель разрешения кадра',
+        'field.viewScaleHint':
+          '1 — нативное, 2 — половина (по умолчанию), 4 — четверть. Меньше пикселей — быстрее кадр.',
+        'tab.title': 'Экран ВМ',
+        'tab.guide': 'Смотреть экран виртуальной машины, пока агент с ней работает.',
+        'view.follow': 'следовать за агентом',
+        'view.pause': 'пауза',
+        'view.play': 'смотреть',
+        'view.snap': 'кадр сейчас',
+        'view.release': 'отключить консоль',
+        'view.released': 'сессия консоли закрыта',
+        'view.releaseFailed': 'закрыть сессию не удалось',
+        'view.fps': 'кадров/с',
+        'view.scale': 'разрешение',
+        'view.scaleNative': 'нативное',
+        'view.scaleHalf': 'половина',
+        'view.scaleQuarter': 'четверть',
+        'view.empty': 'В allowVmids нет ни одной ВМ — добавьте её в настройках плагина.',
+        'view.consoleOff': 'Инструменты консоли выключены настройкой enableConsole.',
+        'view.waiting': 'Ждём первый кадр…',
+        'view.paused': 'поток на паузе',
+        'view.blank': 'Кадр почти одноцветный: гость погасил дисплей или экран заблокирован.',
+        'view.closed': 'сессия консоли закрылась — следующий кадр откроет её заново',
+        'view.status': 'ВМ {vmid} · экран {resolution} · кадр {capture} · курсор {pointer}',
+        'view.bytes': '{kb} КБ',
+        'view.hostError': 'хост ответил ошибкой: {message}',
+        'view.noHost': 'нет связи с хостом: {message}',
+        'view.agentVm': 'агент работал с ВМ {vmid}',
       },
       en: {
         section: 'Proxmox VE',
@@ -120,6 +157,43 @@ window.__ModuleLoader__.load({
         'field.guestTypeCommand': 'Non-ASCII typing in a Linux guest',
         'field.guestTypeCommandHint':
           'Script that types non-ASCII text inside a Linux guest; receives base64 text as $1.',
+        'field.viewAutoOpen': 'Open the screen panel by itself',
+        'field.viewAutoOpenHint':
+          'When the agent touches a VM console, the “VM screen” tab opens itself in the right sidebar.',
+        'field.viewFollowAgent': 'Panel follows the agent',
+        'field.viewFollowAgentHint':
+          'Show the VM the agent is working with. Off — the panel keeps the VM you picked by hand.',
+        'field.viewFps': 'Frames per second in the panel',
+        'field.viewFpsHint':
+          '1–4. Two frames per second is the working pace: the agent’s moves are visible, the node stays calm.',
+        'field.viewScale': 'Frame resolution divisor',
+        'field.viewScaleHint':
+          '1 — native, 2 — half (default), 4 — quarter. Fewer pixels — faster frame.',
+        'tab.title': 'VM screen',
+        'tab.guide': 'Watch the virtual machine screen while the agent works with it.',
+        'view.follow': 'follow the agent',
+        'view.pause': 'pause',
+        'view.play': 'watch',
+        'view.snap': 'frame now',
+        'view.release': 'disconnect console',
+        'view.released': 'console session closed',
+        'view.releaseFailed': 'could not close the session',
+        'view.fps': 'frames/s',
+        'view.scale': 'resolution',
+        'view.scaleNative': 'native',
+        'view.scaleHalf': 'half',
+        'view.scaleQuarter': 'quarter',
+        'view.empty': 'No VM is allowed yet — add one in the plugin settings.',
+        'view.consoleOff': 'Console tools are disabled by the enableConsole setting.',
+        'view.waiting': 'Waiting for the first frame…',
+        'view.paused': 'stream paused',
+        'view.blank': 'The frame is almost one colour: the guest blanked the display or the screen is locked.',
+        'view.closed': 'the console session closed — the next frame opens it again',
+        'view.status': 'VM {vmid} · screen {resolution} · frame {capture} · pointer {pointer}',
+        'view.bytes': '{kb} KB',
+        'view.hostError': 'the host answered with an error: {message}',
+        'view.noHost': 'no connection to the host: {message}',
+        'view.agentVm': 'the agent worked with VM {vmid}',
       },
     };
 
@@ -147,6 +221,10 @@ window.__ModuleLoader__.load({
       { name: 'sessionIdleMs', kind: 'number' },
       { name: 'guestTimeoutMs', kind: 'number' },
       { name: 'guestTypeCommand', kind: 'textarea', hintKey: 'field.guestTypeCommandHint' },
+      { name: 'viewAutoOpen', kind: 'boolean', hintKey: 'field.viewAutoOpenHint' },
+      { name: 'viewFollowAgent', kind: 'boolean', hintKey: 'field.viewFollowAgentHint' },
+      { name: 'viewFps', kind: 'number', hintKey: 'field.viewFpsHint' },
+      { name: 'viewScale', kind: 'number', hintKey: 'field.viewScaleHint' },
     ];
 
     const STYLE_ID = 'dsh-pve-vm-styles';
@@ -167,6 +245,19 @@ window.__ModuleLoader__.load({
 .pve-vm-select, .pve-vm-textarea { width: 100%; font: inherit; font-size: 12.5px; padding: 8px 10px; border-radius: 8px; border: 1px solid var(--dsw-alias-border-l1); background: var(--dsw-alias-bg-layer-1); color: var(--dsw-alias-label-primary); }
 .pve-vm-textarea { min-height: 96px; resize: vertical; }
 .pve-vm-select:focus, .pve-vm-textarea:focus { outline: none; border-color: var(--dsw-alias-brand-primary); }
+.pve-vm-view { display: flex; flex-direction: column; gap: 10px; padding: 10px 10px 16px; color: var(--dsw-alias-label-primary); font-size: 12.5px; }
+.pve-vm-view-head { display: flex; flex-wrap: wrap; align-items: center; gap: 6px; }
+.pve-vm-chip { font: inherit; font-size: 12px; padding: 3px 9px; border-radius: 999px; border: 1px solid var(--dsw-alias-border-l1); background: var(--dsw-alias-bg-layer-2); color: var(--dsw-alias-label-secondary); cursor: pointer; }
+.pve-vm-chip[data-active="1"] { border-color: var(--dsw-alias-brand-primary); color: var(--dsw-alias-brand-primary); }
+.pve-vm-stage { display: flex; align-items: center; justify-content: center; min-height: 110px; border: 1px solid var(--dsw-alias-border-l1); border-radius: 8px; background: var(--dsw-alias-bg-layer-2); overflow: hidden; }
+.pve-vm-stage img { display: block; width: 100%; height: auto; }
+.pve-vm-stage-empty { padding: 22px 14px; text-align: center; color: var(--dsw-alias-label-secondary); font-size: 12px; line-height: 1.5; }
+.pve-vm-line { color: var(--dsw-alias-label-secondary); font-size: 11.5px; line-height: 1.45; }
+.pve-vm-warn { border: 1px solid var(--dsw-alias-state-warn-primary); color: var(--dsw-alias-state-warn-primary); border-radius: 8px; padding: 6px 9px; font-size: 11.5px; line-height: 1.45; }
+.pve-vm-error { border: 1px solid var(--dsw-alias-state-error-primary); color: var(--dsw-alias-state-error-primary); border-radius: 8px; padding: 6px 9px; font-size: 11.5px; line-height: 1.45; }
+.pve-vm-ok { color: var(--dsw-alias-state-success-primary); font-size: 11.5px; }
+.pve-vm-tools { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; }
+.pve-vm-tools .pve-vm-select { width: auto; }
 `;
 
     /** Add the card styles once. */
@@ -364,6 +455,398 @@ window.__ModuleLoader__.load({
       });
     }
 
+    /** The tab type id: it is also the key in the right sidebar slots. */
+    const TAB_ID = 'dsh-pve-vm/console';
+    /** The tab type discriminator: tabs are opened by this name. */
+    const TAB_KIND = 'pve-vm-console';
+    /** Base path of the panel routes on the host (lib/routes.js). */
+    const VIEW_BASE = '/pve-vm/console';
+    /** How often the panel asks the node for its state, ms. */
+    const STATE_POLL_MS = 2500;
+    /** How often the plugin checks "the agent touched a console" for auto-open, ms. */
+    const AUTO_OPEN_POLL_MS = 3000;
+
+    /**
+     * Panel settings from the same form as `settings.section`.
+     * The bounds are the host's `resolveSettings` bounds: a value typed by a human
+     * may be anything, while the frame has to stay predictable.
+     * @param {object} snapshot the configForms snapshot.
+     * @returns {{autoOpen:boolean, followAgent:boolean, fps:number, scale:number}}
+     */
+    function viewDefaults(snapshot) {
+      const values = snapshot?.value ?? {};
+      const clamp = (value, min, max, fallback) => {
+        const parsed = Math.round(Number(value));
+        if (!Number.isFinite(parsed)) return fallback;
+        return Math.max(min, Math.min(max, parsed));
+      };
+      return {
+        autoOpen: values.viewAutoOpen === true,
+        followAgent: values.viewFollowAgent !== false,
+        fps: clamp(values.viewFps, 1, 4, 2),
+        scale: clamp(values.viewScale, 1, 4, 2),
+      };
+    }
+
+    /**
+     * What the panel shows right now. Both the tab body and its chip need it: the
+     * chip is drawn by a separate slot and knows nothing about component state.
+     */
+    const viewStore = {
+      vmid: null,
+      listeners: new Set(),
+      set(vmid) {
+        if (this.vmid === vmid) return;
+        this.vmid = vmid;
+        for (const listener of [...this.listeners]) listener();
+      },
+      subscribe(listener) {
+        this.listeners.add(listener);
+        return () => this.listeners.delete(listener);
+      },
+    };
+
+    /** Frame metadata from the route response headers. */
+    function readFrameMeta(headers) {
+      return {
+        resolution: headers.get('x-pve-vm-resolution'),
+        capture: headers.get('x-pve-vm-capture'),
+        scale: Number(headers.get('x-pve-vm-scale') ?? 1),
+        pointer: headers.get('x-pve-vm-pointer'),
+        blank: headers.get('x-pve-vm-blank') === '1',
+        session: headers.get('x-pve-vm-session'),
+      };
+    }
+
+    /** The tab chip: "VM screen · 121". */
+    function ConsoleTitle(props) {
+      const t = props.t;
+      const [vmid, setVmid] = React.useState(viewStore.vmid);
+      React.useEffect(() => viewStore.subscribe(() => setVmid(viewStore.vmid)), []);
+      return jsx('span', { children: vmid === null ? t('tab.title') : `${t('tab.title')} · ${vmid}` });
+    }
+
+    /**
+     * The observation panel: a live VM screen in the right sidebar.
+     *
+     * Frames come from the plugin's routes (`/pve-vm/console/frame`) — the very
+     * frames the agent sees, taken from its RFB session. The stream runs while the
+     * tab is visible and not paused; the next frame is requested only after the
+     * previous one arrived, so the session queue is never flooded.
+     * @param {{t:Function, ctx:object, form:object, useTabInfo?:Function}} props the slot props.
+     */
+    function ConsolePane(props) {
+      const t = props.t;
+      const form = props.form;
+      const tab = typeof props.useTabInfo === 'function' ? props.useTabInfo() : null;
+      const visible = tab?.tab?.visible !== false;
+
+      const [settings, setSettings] = React.useState(() => viewDefaults(form.getSnapshot()));
+      const [state, setState] = React.useState(null);
+      const [stateError, setStateError] = React.useState(null);
+      const [follow, setFollow] = React.useState(null);
+      const [manualVmid, setManualVmid] = React.useState(null);
+      const [streaming, setStreaming] = React.useState(true);
+      const [fps, setFps] = React.useState(null);
+      const [scale, setScale] = React.useState(null);
+      const [frame, setFrame] = React.useState(null);
+      const [frameError, setFrameError] = React.useState(null);
+      const [note, setNote] = React.useState(null);
+      const [tick, setTick] = React.useState(0);
+      const frameUrl = React.useRef(null);
+      const [, setLocaleTick] = React.useState(0);
+
+      React.useEffect(() => form.subscribe(() => setSettings(viewDefaults(form.getSnapshot()))), [form]);
+      // A language change must repaint the panel labels, not just its title.
+      React.useEffect(() => {
+        const unsubscribe = props.ctx.locale.subscribe?.(() => setLocaleTick((value) => value + 1));
+        return typeof unsubscribe === 'function' ? unsubscribe : undefined;
+      }, [props.ctx]);
+
+      // Node state: the allowed VMs, live sessions and whom to follow.
+      React.useEffect(() => {
+        if (!visible) return undefined;
+        let cancelled = false;
+        let timer;
+        const poll = async () => {
+          try {
+            const response = await fetch(`${VIEW_BASE}/state`, {
+              headers: { accept: 'application/json' },
+              cache: 'no-store',
+            });
+            const body = await response.json().catch(() => ({}));
+            if (cancelled) return;
+            if (response.ok) {
+              setState(body);
+              setStateError(null);
+            } else {
+              setStateError(body.error ?? `HTTP ${response.status}`);
+            }
+          } catch (error) {
+            if (!cancelled) setStateError(String(error?.message ?? error));
+          } finally {
+            if (!cancelled) timer = setTimeout(poll, STATE_POLL_MS);
+          }
+        };
+        poll();
+        return () => {
+          cancelled = true;
+          clearTimeout(timer);
+        };
+      }, [visible]);
+
+      const followAgent = follow ?? settings.followAgent;
+      const fpsEff = fps ?? settings.fps;
+      const scaleEff = scale ?? settings.scale;
+      const allowed = Array.isArray(state?.allowed) ? state.allowed : [];
+      const lastVmid = state?.last?.vmid ?? null;
+      const preferred = followAgent ? lastVmid : manualVmid;
+      const vmid = allowed.includes(preferred)
+        ? preferred
+        : allowed.includes(manualVmid)
+          ? manualVmid
+          : (allowed[0] ?? null);
+
+      React.useEffect(() => viewStore.set(vmid), [vmid]);
+
+      // The frame stream: the next request only after the previous answer.
+      React.useEffect(() => {
+        if (!visible || !streaming || vmid === null) return undefined;
+        let cancelled = false;
+        let timer;
+        const step = async () => {
+          const started = Date.now();
+          try {
+            const response = await fetch(`${VIEW_BASE}/frame?vmid=${vmid}&scale=${scaleEff}&cursor=1`, {
+              cache: 'no-store',
+            });
+            if (!response.ok) {
+              const body = await response.json().catch(() => ({}));
+              throw new Error(body.error ?? `HTTP ${response.status}`);
+            }
+            const blob = await response.blob();
+            if (cancelled) return;
+            const next = URL.createObjectURL(blob);
+            if (frameUrl.current !== null) URL.revokeObjectURL(frameUrl.current);
+            frameUrl.current = next;
+            setFrame({ url: next, meta: readFrameMeta(response.headers), bytes: blob.size, at: Date.now() });
+            setFrameError(null);
+          } catch (error) {
+            if (!cancelled) setFrameError(String(error?.message ?? error));
+          } finally {
+            if (!cancelled) {
+              const wait = Math.max(80, Math.round(1000 / fpsEff) - (Date.now() - started));
+              timer = setTimeout(step, wait);
+            }
+          }
+        };
+        step();
+        return () => {
+          cancelled = true;
+          clearTimeout(timer);
+        };
+      }, [visible, streaming, vmid, scaleEff, fpsEff, tick]);
+
+      // The last frame lives in a blob URL: it must be released when the panel unloads.
+      React.useEffect(
+        () => () => {
+          if (frameUrl.current !== null) URL.revokeObjectURL(frameUrl.current);
+        },
+        [],
+      );
+
+      /** Close the VM console session on the node. */
+      const release = async () => {
+        try {
+          const response = await fetch(`${VIEW_BASE}/release`, {
+            method: 'POST',
+            headers: { 'content-type': 'application/json' },
+            body: JSON.stringify({ vmid }),
+          });
+          if (!response.ok) throw new Error(`HTTP ${response.status}`);
+          setNote(t('view.released'));
+        } catch {
+          setNote(t('view.releaseFailed'));
+        }
+        setTimeout(() => setNote(null), 2500);
+      };
+
+      const chips = allowed.map((id) =>
+        jsx(
+          'button',
+          {
+            type: 'button',
+            className: 'pve-vm-chip',
+            'data-active': id === vmid ? '1' : '0',
+            onClick: () => {
+              setManualVmid(id);
+              setFollow(false);
+            },
+            children: String(id),
+          },
+          id,
+        ),
+      );
+
+      const banners = [];
+      if (stateError !== null) {
+        banners.push(jsx('div', { className: 'pve-vm-error', children: t('view.noHost', { message: stateError }) }, 'state'));
+      }
+      if (state?.consoleEnabled === false) {
+        banners.push(jsx('div', { className: 'pve-vm-warn', children: t('view.consoleOff') }, 'console'));
+      }
+      if (allowed.length === 0 && state !== null) {
+        banners.push(jsx('div', { className: 'pve-vm-warn', children: t('view.empty') }, 'empty'));
+      }
+      if (frameError !== null) {
+        banners.push(jsx('div', { className: 'pve-vm-error', children: t('view.hostError', { message: frameError }) }, 'frame'));
+      }
+      if (frame?.meta?.blank === true) {
+        banners.push(jsx('div', { className: 'pve-vm-warn', children: t('view.blank') }, 'blank'));
+      }
+      if (frame?.meta?.session === 'closed') {
+        banners.push(jsx('div', { className: 'pve-vm-warn', children: t('view.closed') }, 'closed'));
+      }
+      if (!streaming) {
+        banners.push(jsx('div', { className: 'pve-vm-line', children: t('view.paused') }, 'paused'));
+      }
+
+      const status =
+        frame === null
+          ? null
+          : `${t('view.status', {
+              vmid: String(vmid ?? '—'),
+              resolution: frame.meta.resolution ?? '—',
+              capture: frame.meta.capture ?? '—',
+              pointer: frame.meta.pointer ?? '—',
+            })} · ${t('view.bytes', { kb: String(Math.max(1, Math.round(frame.bytes / 1024))) })}`;
+
+      return jsxs('div', {
+        className: 'pve-vm-view',
+        children: [
+          jsxs('div', {
+            className: 'pve-vm-view-head',
+            children: [
+              ...chips,
+              lastVmid !== null
+                ? jsx('span', { className: 'pve-vm-line', children: t('view.agentVm', { vmid: String(lastVmid) }) })
+                : null,
+            ],
+          }),
+          jsx('div', {
+            className: 'pve-vm-stage',
+            children:
+              frame === null
+                ? jsx('div', {
+                    className: 'pve-vm-stage-empty',
+                    children: allowed.length === 0 ? t('view.empty') : t('view.waiting'),
+                  })
+                : jsx('img', { src: frame.url, alt: '' }),
+          }),
+          status === null ? null : jsx('div', { className: 'pve-vm-line', children: status }),
+          ...banners,
+          jsxs('div', {
+            className: 'pve-vm-tools',
+            children: [
+              jsx(Button, {
+                variant: 'outline',
+                size: 'sm',
+                children: streaming ? t('view.pause') : t('view.play'),
+                onClick: () => setStreaming((value) => !value),
+              }),
+              jsx(Button, {
+                variant: 'ghost',
+                size: 'sm',
+                disabled: vmid === null,
+                children: t('view.snap'),
+                onClick: () => setTick((value) => value + 1),
+              }),
+              jsx(Button, {
+                variant: 'ghost',
+                size: 'sm',
+                disabled: vmid === null,
+                children: t('view.release'),
+                onClick: release,
+              }),
+              jsx(Checkbox, {
+                checked: followAgent,
+                label: t('view.follow'),
+                onChange: (next) => setFollow(next),
+              }),
+              jsx('select', {
+                className: 'pve-vm-select',
+                value: String(fpsEff),
+                title: t('view.fps'),
+                onChange: (event) => setFps(Number(event.target.value)),
+                children: [1, 2, 3, 4].map((value) =>
+                  jsx('option', { value: String(value), children: `${value} ${t('view.fps')}` }, value),
+                ),
+              }),
+              jsx('select', {
+                className: 'pve-vm-select',
+                value: String(scaleEff),
+                title: t('view.scale'),
+                onChange: (event) => setScale(Number(event.target.value)),
+                children: [
+                  { value: 1, key: 'view.scaleNative' },
+                  { value: 2, key: 'view.scaleHalf' },
+                  { value: 4, key: 'view.scaleQuarter' },
+                ].map((item) =>
+                  jsx('option', { value: String(item.value), children: `${t('view.scale')}: ${t(item.key)}` }, item.value),
+                ),
+              }),
+              note === null ? null : jsx('span', { className: 'pve-vm-ok', children: note }),
+            ],
+          }),
+        ],
+      });
+    }
+
+    /**
+     * Tab auto-open: when the agent touches a VM console the panel reveals itself —
+     * if the owner asked for it with the `viewAutoOpen` setting.
+     *
+     * The first state read only remembers the mark: otherwise every page reload
+     * would open the tab because of the agent's older work.
+     * @param {object} ctx the client context.
+     * @returns {() => void} stops watching.
+     */
+    function armAutoOpen(ctx) {
+      const form = ctx.configForms.get(NAMESPACE);
+      let primed = false;
+      let seenAt = 0;
+      let stopped = false;
+      const tick = async () => {
+        if (stopped) return;
+        const values = form.getSnapshot()?.value ?? {};
+        if (values.viewAutoOpen !== true) return;
+        try {
+          const response = await fetch(`${VIEW_BASE}/state`, { headers: { accept: 'application/json' }, cache: 'no-store' });
+          if (!response.ok) return;
+          const state = await response.json();
+          const at = Number(state?.last?.at ?? 0);
+          if (!primed) {
+            primed = true;
+            seenAt = at;
+            return;
+          }
+          if (at <= seenAt) return;
+          seenAt = at;
+          const sessionId = ctx.sidebarRight.mounted.getSnapshot();
+          if (sessionId === undefined) return;
+          ctx.sidebarRight.openTabIn(sessionId, TAB_KIND);
+        } catch {
+          // Network and response parsing are no reason to shout in the console: the
+          // panel shows the error anyway once it is opened.
+        }
+      };
+      const timer = setInterval(tick, AUTO_OPEN_POLL_MS);
+      return () => {
+        stopped = true;
+        clearInterval(timer);
+      };
+    }
+
     /**
      * Attach the card to the settings section.
      * @param {object} ctx the client context.
@@ -393,6 +876,58 @@ window.__ModuleLoader__.load({
           PveVmSection,
         ),
       );
+      // The observation panel is a second, independent contribution. Not every
+      // composition has a right sidebar, so the services are declared through
+      // ctx.inject: without them the settings card still works (see §4.4.1 of the
+      // project AGENTS.md — never turn a service read into an undeclared, optional
+      // dependency).
+      // каждой композиции, поэтому сервисы объявлены через ctx.inject: без них
+      // карточка настроек всё равно работает (см. §4.4.1 проектного AGENTS.md —
+      // не превращать чтение сервиса в необязательную, но необъявленную зависимость).
+      ctx.inject(['sidebarRight', 'sidebarRightTabs'], (scoped) => {
+        const disposers = [];
+        const own = (dispose) => {
+          if (typeof dispose === 'function') disposers.push(dispose);
+        };
+
+        own(
+          scoped.sidebarRightTabs.register({
+            id: TAB_ID,
+            kind: TAB_KIND,
+            title: () => translate()('tab.title'),
+            guide: [
+              {
+                id: 'console',
+                order: 55,
+                title: () => translate()('tab.title'),
+                description: () => translate()('tab.guide'),
+              },
+            ],
+          }),
+        );
+
+        const injectPane = () => ({ ctx, form: ctx.configForms.get(NAMESPACE) });
+        own(
+          scoped.slots.inject('sidebar.right.pane.tab', () =>
+            scoped.slots.register(
+              { name: 'sidebar.right.pane.tab', key: TAB_ID, locale: LOCALE_NS, inject: injectPane },
+              ConsolePane,
+            ),
+          ),
+        );
+        own(
+          scoped.slots.inject('sidebar.right.pane.tab.title', () =>
+            scoped.slots.register({ name: 'sidebar.right.pane.tab.title', key: TAB_ID, locale: LOCALE_NS }, ConsoleTitle),
+          ),
+        );
+
+        // Автооткрытие — отдельный слой: он живёт и когда панель закрыта.
+        const stopAutoOpen = armAutoOpen(ctx);
+        return () => {
+          stopAutoOpen();
+          for (const dispose of disposers.reverse()) dispose();
+        };
+      });
     }
 
     exports.apply = apply;

package/index.js

@@ -11,6 +11,8 @@
  */
 import { Config, resolveSettings, snapshot } from './lib/config.js';
 import { VmController } from './lib/session.js';
+import { ConsoleView } from './lib/view.js';
+import { registerViewRoutes } from './lib/routes.js';
 import { consoleTools } from './tools/console.js';
 import { guestTools } from './tools/guest.js';
 
@@ -35,6 +37,13 @@ export function apply(ctx, config) {
   const deps = { controller, settings, attachments: ctx.attachments };
   for (const tool of [...consoleTools(deps), ...guestTools(deps)]) ctx.tools.register(tool);
 
+  // Observation panel: the client half pulls frames from these routes. A
+  // composition may have no web server at all (headless profile) — then there is
+  // simply no panel while the tools keep working, so the dependency is declared
+  // through ctx.inject instead of the plugin's mandatory `inject` list.
+  const view = new ConsoleView({ controller, settings });
+  ctx.inject(['webServer'], (scoped) => registerViewRoutes(scoped.webServer, { view }));
+
   // The computer-use slot is per composition: if another provider has already taken it,
   // the plugin tools still work, so the error is only recorded.
   ctx.inject(['computerUse'], (scoped) => {

package/lib/config.js

@@ -56,10 +56,16 @@ export const CONFIG_KEYS = [
   'allowVmids',
   'enableConsole',
   'enableGuest',
+  'inputChannel',
+  'sendkeyDelayMs',
   'consoleTimeoutMs',
   'sessionIdleMs',
   'guestTimeoutMs',
   'guestTypeCommand',
+  'viewAutoOpen',
+  'viewFollowAgent',
+  'viewFps',
+  'viewScale',
 ];
 
 /** Settings schema: the id of the entry in `cordis.patch.yml` must match `pve-vm`. */
@@ -141,6 +147,36 @@ export const Config = z.object({
       'Script for typing non-ASCII text in a Linux guest: takes the base64 text as its first argument ($1).',
     )
     .volatile(),
+  viewAutoOpen: z
+    .boolean()
+    .default(false)
+    .description(
+      'Open the "VM screen" tab in the right sidebar by itself when the agent touches a VM console.',
+    )
+    .volatile(),
+  viewFollowAgent: z
+    .boolean()
+    .default(true)
+    .description('The panel follows the VM the agent is working with by default.')
+    .volatile(),
+  viewFps: z
+    .number()
+    .step(1)
+    .min(1)
+    .max(4)
+    .default(2)
+    .description('How many frames per second the panel asks the node for (1–4).')
+    .volatile(),
+  viewScale: z
+    .number()
+    .step(1)
+    .min(1)
+    .max(4)
+    .default(2)
+    .description(
+      'Frame resolution divisor in the panel: 1 — native, 2 — half (default), 4 — quarter.',
+    )
+    .volatile(),
 });
 
 /**
@@ -169,7 +205,7 @@ function expandHome(path) {
 /**
  * Reduce the settings slice to a shape convenient for the tools.
  * @param {Record<string, unknown>} section the result of {@link snapshot}.
- * @returns {{ssh:object,allowVmids:number[],enableConsole:boolean,enableGuest:boolean,consoleTimeoutMs:number,sessionIdleMs:number,guestTimeoutMs:number,guestTypeCommand:string}}
+ * @returns {{ssh:object,allowVmids:number[],enableConsole:boolean,enableGuest:boolean,consoleTimeoutMs:number,sessionIdleMs:number,guestTimeoutMs:number,guestTypeCommand:string,view:{autoOpen:boolean,followAgent:boolean,fps:number,scale:number}}}
  */
 export function resolveSettings(section) {
   const allowVmids = Array.isArray(section.allowVmids)
@@ -193,5 +229,25 @@ export function resolveSettings(section) {
     sessionIdleMs: Number(section.sessionIdleMs ?? 300000),
     guestTimeoutMs: Number(section.guestTimeoutMs ?? 60000),
     guestTypeCommand: typeof section.guestTypeCommand === 'string' ? section.guestTypeCommand : DEFAULT_GUEST_TYPE_COMMAND,
+    view: {
+      autoOpen: section.viewAutoOpen === true,
+      followAgent: section.viewFollowAgent !== false,
+      fps: clampNumber(section.viewFps, 1, 4, 2),
+      scale: clampNumber(section.viewScale, 1, 4, 2),
+    },
   };
 }
+
+/**
+ * An integer inside bounds (for the panel fields: fps and the resolution divisor).
+ * @param {unknown} value live settings value.
+ * @param {number} min lower bound.
+ * @param {number} max upper bound.
+ * @param {number} fallback default value.
+ * @returns {number} a usable number.
+ */
+function clampNumber(value, min, max, fallback) {
+  const parsed = Math.round(Number(value));
+  if (!Number.isFinite(parsed)) return fallback;
+  return Math.max(min, Math.min(max, parsed));
+}

package/lib/png.js

@@ -106,6 +106,25 @@ export function downscaleBox(image, factor) {
   return { width, height, data: out, scale: 1 / step };
 }
 
+/**
+ * An almost single-colour frame: most often a blanked display or a black lock screen.
+ * The check samples coarsely instead of scanning a million pixels. It lives here, not in
+ * the tools, because the observation panel needs the very same verdict.
+ * @param {{width:number,height:number,data:Uint8Array}} image RGBA frame.
+ * @returns {boolean} true when the frame is practically one colour.
+ */
+export function looksBlank(image) {
+  const step = Math.max(4, Math.floor((image.width * image.height) / 4096)) * 4;
+  let min = 255;
+  let max = 0;
+  for (let i = 0; i < image.data.length; i += step) {
+    const value = image.data[i];
+    if (value < min) min = value;
+    if (value > max) max = value;
+  }
+  return max - min <= 6;
+}
+
 /**
  * Rectangle cropping; the coordinates are clipped to the frame.
  * @param {{width:number,height:number,data:Uint8Array}} image the source RGBA frame.

package/lib/rfb.js

@@ -350,19 +350,38 @@ export class RfbClient extends EventEmitter {
 
   /**
    * Read messages until the next FramebufferUpdate.
-   * @param {{timeoutMs?:number}} [options] wait timeout.
-   * @returns {Promise<{rectangles:number, bytes:number, resized:boolean, cursor:boolean, bell:boolean}>}
+   *
+   * `idleOk` is what the observation panel needs: it asks for an incremental frame
+   * often, and "nothing changed" is a normal answer from QEMU (it simply stays
+   * silent). Silence at a message boundary is safe — the current framebuffer is
+   * returned. A timeout in the middle of a message means half a frame was lost,
+   * which is a real error and the session has to be recreated.
+   * @param {{timeoutMs?:number, idleOk?:boolean}} [options] wait timeout and tolerance for silence.
+   * @returns {Promise<{rectangles:number, bytes:number, resized:boolean, cursor:boolean, bell:boolean, idle?:boolean}>}
    */
   async waitForUpdate(options = {}) {
     const deadline = Date.now() + (options.timeoutMs ?? this.timeoutMs);
+    const idleOk = options.idleOk === true;
     let rectangles = 0;
     let cursorUpdated = false;
     let bell = false;
     let resized = false;
     for (;;) {
       const left = deadline - Date.now();
-      if (left <= 0) throw new RfbError('timeout waiting for an RFB frame', 'TIMEOUT');
-      const type = (await this.reader.exact(1, left))[0];
+      if (left <= 0) {
+        if (idleOk) return { rectangles: 0, bytes: 0, resized: false, cursor: false, bell: false, idle: true };
+        throw new RfbError('timeout waiting for an RFB frame', 'TIMEOUT');
+      }
+      let first;
+      try {
+        first = await this.reader.exact(1, left);
+      } catch (error) {
+        if (idleOk && error?.code === 'TIMEOUT') {
+          return { rectangles: 0, bytes: 0, resized: false, cursor: false, bell: false, idle: true };
+        }
+        throw error;
+      }
+      const type = first[0];
       if (type === 0) {
         const header = await this.reader.exact(3, left);
         const count = header.readUInt16BE(1);

package/lib/routes.js

@@ -0,0 +1,197 @@
+/**
+ * HTTP routes of the observation panel: `/pve-vm/console/*`.
+ *
+ * Why HTTP and not a harness RPC. The panel is an ordinary page of the same origin,
+ * and frames are easy to pull with `fetch`; the browser-skill overview is built the
+ * same way (`/bsk-observation/*`). The routes register on the `webServer` service; a
+ * composition without a web server (a headless profile) simply never registers them,
+ * while the agent tools keep working as usual.
+ *
+ * Security. A VM screen is sensitive: passwords, chats, a working desktop. So these
+ * routes carry their own fence: we answer only when Host is a loopback authority
+ * (localhost / 127.0.0.0/8 / [::1]), a present Origin matches Host, and
+ * `sec-fetch-site: cross-site` is refused outright; POST must be `application/json`,
+ * so a cross-site simple request never reaches the handler. DSH itself applies the
+ * same order of checks to `/api` (`@deepseek-ai/dsh-client-connection`), and so does
+ * browser-skill; it is repeated here so that the published plugin depends on no
+ * internal package.
+ */
+import { VmError } from './session.js';
+
+/** Base path of the panel routes. */
+export const VIEW_BASE = '/pve-vm/console';
+
+/** Error codes to HTTP statuses. */
+const STATUS_BY_CODE = {
+  BAD_VMID: 400,
+  NOT_ALLOWED: 403,
+  CHANNEL_DISABLED: 403,
+  CONSOLE_CLOSED: 409,
+  CONSOLE_FAILED: 502,
+  SSH_FAILED: 502,
+  EOF: 502,
+  TIMEOUT: 504,
+};
+
+/** Answer with JSON. */
+function sendJson(res, status, body) {
+  const payload = Buffer.from(JSON.stringify(body), 'utf8');
+  res.writeHead(status, {
+    'content-type': 'application/json; charset=utf-8',
+    'content-length': String(payload.length),
+    'cache-control': 'no-store',
+  });
+  res.end(payload);
+}
+
+/** Turn a panel request failure into a response. */
+function fail(res, error) {
+  if (res.headersSent) {
+    res.end();
+    return;
+  }
+  const code = error instanceof VmError ? error.code : undefined;
+  const status = STATUS_BY_CODE[code] ?? 500;
+  sendJson(res, status, { error: error?.message ?? String(error), ...(code ? { code } : {}) });
+}
+
+/**
+ * The fence's verdict: the reason for refusal, or undefined.
+ * @param {import('node:http').IncomingMessage} req the request.
+ * @returns {string|undefined} the reason for refusal.
+ */
+function fenceViolation(req) {
+  const host = req.headers.host ?? '';
+  const hostname = /^\[.*\](?::\d+)?$/.test(host) ? host.slice(1, host.indexOf(']')) : host.split(':')[0];
+  const loopback =
+    hostname === 'localhost' ||
+    hostname.endsWith('.localhost') ||
+    hostname === '::1' ||
+    /^127\.\d{1,3}\.\d{1,3}\.\d{1,3}$/.test(hostname);
+  if (!loopback) return 'host is not a loopback authority';
+  const origin = req.headers.origin;
+  if (origin !== undefined && origin !== 'null') {
+    let originHost;
+    try {
+      originHost = new URL(origin).host;
+    } catch {
+      return 'unparseable Origin header';
+    }
+    if (originHost !== host) return 'Origin does not match Host';
+  }
+  if (req.headers['sec-fetch-site'] === 'cross-site') return 'sec-fetch-site: cross-site';
+  if (req.method === 'POST') {
+    const contentType = req.headers['content-type'] ?? '';
+    if (!/^\s*application\/json\s*(;|$)/.test(contentType)) return 'POST requires an application/json body';
+  }
+  return undefined;
+}
+
+/** Run the fence; true means the request was already refused and answered. */
+function fenceRejected(req, res) {
+  const violation = fenceViolation(req);
+  if (violation === undefined) return false;
+  sendJson(res, 403, { error: `forbidden: ${violation}` });
+  return true;
+}
+
+/** Read the JSON body of a small request. */
+async function readJsonBody(req) {
+  const chunks = [];
+  let size = 0;
+  for await (const chunk of req) {
+    size += chunk.length;
+    if (size > 64 * 1024) throw new VmError('request body is too large', 'BAD_BODY');
+    chunks.push(chunk);
+  }
+  if (size === 0) return {};
+  try {
+    return JSON.parse(Buffer.concat(chunks).toString('utf8'));
+  } catch {
+    throw new VmError('request body is not JSON', 'BAD_BODY');
+  }
+}
+
+/**
+ * Register the panel routes on the web server.
+ * @param {object} webServer the `webServer` service.
+ * @param {{view: object}} deps the observation panel.
+ * @returns {() => void} disposer removing the routes.
+ */
+export function registerViewRoutes(webServer, deps) {
+  const view = deps.view;
+  const disposers = [
+    webServer.register({
+      kind: 'exact',
+      path: `${VIEW_BASE}/state`,
+      handler: (req, res) => {
+        if (req.method !== 'GET') {
+          sendJson(res, 405, { error: 'method not allowed' });
+          return;
+        }
+        if (fenceRejected(req, res)) return;
+        try {
+          sendJson(res, 200, view.state());
+        } catch (error) {
+          fail(res, error);
+        }
+      },
+    }),
+    webServer.register({
+      kind: 'exact',
+      path: `${VIEW_BASE}/frame`,
+      handler: async (req, res) => {
+        if (req.method !== 'GET') {
+          sendJson(res, 405, { error: 'method not allowed' });
+          return;
+        }
+        if (fenceRejected(req, res)) return;
+        const url = new URL(req.url ?? '/', 'http://localhost');
+        const rawScale = url.searchParams.get('scale');
+        try {
+          const { png, meta } = await view.frame(Number(url.searchParams.get('vmid')), {
+            scale: rawScale === null ? undefined : Number(rawScale),
+            cursor: url.searchParams.get('cursor') !== '0',
+          });
+          res.writeHead(200, {
+            'content-type': 'image/png',
+            'content-length': String(png.length),
+            'cache-control': 'no-store, max-age=0',
+            'x-pve-vm-vmid': String(meta.vmid),
+            'x-pve-vm-resolution': `${meta.width}x${meta.height}`,
+            'x-pve-vm-capture': `${meta.captureWidth}x${meta.captureHeight}`,
+            'x-pve-vm-scale': String(meta.scale),
+            'x-pve-vm-pointer': `${meta.pointerX},${meta.pointerY}`,
+            'x-pve-vm-cursor': meta.cursorDrawn ? '1' : '0',
+            'x-pve-vm-blank': meta.blank ? '1' : '0',
+            'x-pve-vm-session': meta.sessionAlive ? 'alive' : 'closed',
+          });
+          res.end(png);
+        } catch (error) {
+          fail(res, error);
+        }
+      },
+    }),
+    webServer.register({
+      kind: 'exact',
+      path: `${VIEW_BASE}/release`,
+      handler: async (req, res) => {
+        if (req.method !== 'POST') {
+          sendJson(res, 405, { error: 'method not allowed' });
+          return;
+        }
+        if (fenceRejected(req, res)) return;
+        try {
+          const body = await readJsonBody(req);
+          const closed = view.release(Number(body.vmid));
+          sendJson(res, 200, { closed });
+        } catch (error) {
+          fail(res, error);
+        }
+      },
+    }),
+  ];
+  return () => {
+    for (const dispose of disposers.reverse()) dispose();
+  };
+}

package/lib/session.js

@@ -55,10 +55,20 @@ export class VmController {
     this.sessions = new Map();
     /** @type {Map<number, string>} */
     this.osCache = new Map();
+    /** The last VM whose console the agent touched: the panel auto-opens on it. */
+    this.lastConsole = null;
     this.reaper = setInterval(() => this.#reapIdle(), 30000);
     this.reaper.unref?.();
   }
 
+  /**
+   * The last console use: the observation panel shows this VM.
+   * @returns {{vmid:number, at:number}|null} vmid and time (ms), or null.
+   */
+  lastConsoleUse() {
+    return this.lastConsole;
+  }
+
   /** Validate the VM id against the allowlist and the channel switches. */
   assertAllowed(vmid, channel) {
     const settings = this.deps.settings();
@@ -275,12 +285,19 @@ export class VmController {
    * Run an operation with exclusive access to the VM console.
    * @param {number} vmid VM id.
    * @param {(client: RfbClient, session: object) => Promise<T>} operation the action.
-   * @param {{signal?: AbortSignal}} [options]
+   * @param {{signal?: AbortSignal, mark?:boolean}} [options] cancellation and the "the agent worked with this VM" mark.
    * @returns {Promise<T>}
    * @template T
    */
   async withConsole(vmid, operation, options = {}) {
     const id = this.assertAllowed(vmid, 'console');
+    // A mark for the observation panel: it follows the VM the agent works with.
+    // The panel watches frames through this very method, so it has to suppress the
+    // mark (`mark: false`) — otherwise watching declares itself agent work and the
+    // "follow the agent" mode jumps between VMs (seen on a live panel 2026-10-13:
+    // right after a restart `last` named the VM the owner was watching, while no
+    // tool had touched it).
+    if (options.mark !== false) this.lastConsole = { vmid: id, at: Date.now() };
     const session = await this.#consoleSession(id, options.signal);
     const run = async () => {
       if (!session.alive) {

package/lib/view.js

@@ -0,0 +1,145 @@
+/**
+ * The live VM screen for the observation panel.
+ *
+ * Why a separate module. The agent's tools capture a frame on demand and hand it
+ * back as an attachment; a panel needs something else: often, cheaply, and without
+ * attachments — "what is on screen right now". The frame comes from **the same**
+ * RFB session the tools use (`VmController.withConsole`), therefore:
+ *
+ *   • no second VNC session to the VM appears, and the `qm vncproxy` password is
+ *     not reset under the agent's feet;
+ *   • there is still exactly one session queue — the panel and the agent never
+ *     tear the stream for each other;
+ *   • a session the panel holds open is not reaped by the idle reaper (every
+ *     capture refreshes `lastUsed`).
+ *
+ * Frames are requested incrementally: when the screen did not change QEMU stays
+ * silent and the panel gets the current framebuffer from the client's memory (see
+ * `idleOk` in `RfbClient.waitForUpdate`).
+ */
+import { encodePng, downscaleBox, looksBlank } from './png.js';
+
+/** Lower bound on waiting for QEMU: below it we simply cannot read the answer. */
+const MIN_WAIT_MS = 120;
+/**
+ * Upper bound on waiting. This is exactly how long the panel holds the session
+ * queue when the screen does not change (QEMU stays silent on an incremental
+ * request) — and therefore how long the agent waits for its own operation.
+ * 250 ms is enough: QEMU hands over changed regions right after the request, and
+ * anything that happens later is picked up by the next frame.
+ */
+const MAX_WAIT_MS = 250;
+/** zlib level for panel frames: speed over size, the channel is loopback. */
+const PNG_LEVEL = 3;
+/** Allowed resolution divisors (1 — native). */
+const DIVISORS = [1, 2, 3, 4];
+
+/**
+ * How long to wait for a frame at the panel's pace.
+ * @param {number} fps frames per second from the settings.
+ * @returns {number} wait timeout, ms.
+ */
+function waitForFps(fps) {
+  const interval = 1000 / Math.max(1, fps);
+  return Math.max(MIN_WAIT_MS, Math.min(MAX_WAIT_MS, Math.round(interval - 50)));
+}
+
+/**
+ * One VM's screen as the agent sees it.
+ */
+export class ConsoleView {
+  /**
+   * @param {{controller: object, settings: () => object}} deps the VM controller and live settings.
+   */
+  constructor(deps) {
+    this.deps = deps;
+    /** @type {Map<string, Promise<object>>} captures already running: two clients do not queue twice. */
+    this.pending = new Map();
+  }
+
+  /** State for the panel: what is allowed, what is alive, whom to follow. */
+  state() {
+    const settings = this.deps.settings();
+    const last = this.deps.controller.lastConsoleUse();
+    return {
+      allowed: settings.allowVmids,
+      consoleEnabled: settings.enableConsole,
+      sessions: this.deps.controller.listSessions(),
+      last: last ? { vmid: last.vmid, at: last.at } : null,
+      view: settings.view,
+    };
+  }
+
+  /**
+   * A frame of the VM screen as PNG.
+   * @param {number} vmid VM id.
+   * @param {{scale?:number, cursor?:boolean}} [options] resolution divisor and whether to draw the cursor.
+   * @returns {Promise<{png:Buffer, meta:object}>} the PNG and its metadata.
+   */
+  frame(vmid, options = {}) {
+    const settings = this.deps.settings();
+    const id = this.deps.controller.assertAllowed(vmid, 'console');
+    const divisor = DIVISORS.includes(Number(options.scale)) ? Number(options.scale) : settings.view.scale;
+    const drawCursor = options.cursor !== false;
+    const key = `${id}:${divisor}:${drawCursor ? 1 : 0}`;
+    const running = this.pending.get(key);
+    if (running !== undefined) return running;
+    const task = this.#capture(id, divisor, drawCursor, waitForFps(settings.view.fps)).finally(() => {
+      this.pending.delete(key);
+    });
+    this.pending.set(key, task);
+    return task;
+  }
+
+  /**
+   * Close the VM console session (the panel's "disconnect" button).
+   * @param {number} vmid VM id.
+   * @returns {boolean} true when a session existed and was closed.
+   */
+  release(vmid) {
+    return this.deps.controller.close(vmid);
+  }
+
+  /**
+   * Capture a frame with exclusive access to the console.
+   * @param {number} vmid VM id.
+   * @param {number} divisor resolution divisor.
+   * @param {boolean} drawCursor whether to draw the cursor.
+   * @param {number} waitMs how long to wait for the incremental answer.
+   * @returns {Promise<{png:Buffer, meta:object}>}
+   */
+  async #capture(vmid, divisor, drawCursor, waitMs) {
+    // `mark: false` — watching is not agent work: otherwise the panel would move
+    // the "whom to follow" mark on every frame.
+    return this.deps.controller.withConsole(
+      vmid,
+      async (client, session) => {
+        // The first frame of a session is requested in full: a fresh client has no
+        // "changed" regions yet, and an incremental request could stay silent.
+        client.requestUpdate({ incremental: client.bytesIn > 0 });
+        await client.waitForUpdate({ timeoutMs: waitMs, idleOk: true });
+        const frame = client.snapshot({ drawCursor });
+        const scaled = divisor > 1 ? downscaleBox(frame, divisor) : { ...frame, scale: 1 };
+        const png = encodePng(scaled, { level: PNG_LEVEL });
+        return {
+          png,
+          meta: {
+            vmid,
+            width: frame.width,
+            height: frame.height,
+            captureWidth: scaled.width,
+            captureHeight: scaled.height,
+            scale: divisor,
+            pointerX: client.pointer.x,
+            pointerY: client.pointer.y,
+            cursorDrawn: Boolean(client.cursor) && drawCursor,
+            blank: looksBlank(frame),
+            bytes: png.length,
+            sessionAlive: session.alive !== false,
+          },
+        };
+      },
+      { mark: false },
+    );
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-pve-vm",
-  "version": "0.1.1",
+  "version": "0.2.0",
   "description": "Proxmox VE VM control for DeepSeek Harness: see the screen and click in a VM console over VNC, plus commands, files and typed text through the QEMU guest agent.",
   "type": "module",
   "license": "MIT",
@@ -58,6 +58,7 @@
       "inject": [
         "@deepseek-ai/dsh-client-ui-settings",
         "@deepseek-ai/dsh-client-ui-slots",
+        "@deepseek-ai/dsh-client-ui-sidebar-right",
         "@deepseek-ai/dsh-client-locale"
       ]
     }
@@ -74,7 +75,7 @@
     "@deepseek-ai/schemastery": "^3.18.4"
   },
   "scripts": {
-    "check": "node --check index.js && node --check client.js && node --check lib/config.js && node --check lib/guest.js && node --check lib/hmp.js && node --check lib/keys.js && node --check lib/png.js && node --check lib/rfb.js && node --check lib/session.js && node --check lib/ssh.js && node --check tools/console.js && node --check tools/guest.js",
+    "check": "node --check index.js && node --check client.js && node --check lib/config.js && node --check lib/guest.js && node --check lib/hmp.js && node --check lib/keys.js && node --check lib/png.js && node --check lib/rfb.js && node --check lib/routes.js && node --check lib/session.js && node --check lib/ssh.js && node --check lib/view.js && node --check tools/console.js && node --check tools/guest.js",
     "test": "node tools/check-plugin.mjs && node tools/check-client.mjs",
     "prepublishOnly": "npm run check",
     "probe": "node tools/probe-rfb.mjs"