sensorium-mcp 3.0.76 → 3.0.78

package/templates/sensorium-repair.default.md

@@ -0,0 +1,99 @@
+---
+name: Sensorium Self-Repair
+triggers:
+  - debug sensorium
+  - fix sensorium
+  - repair sensorium
+  - sensorium crashing
+  - sensorium is crashing
+  - sensorium down
+  - sensorium broken
+  - mcp crashing
+  - mcp is dead
+  - supervisor loop
+  - restart loop
+  - emergency repair
+  - sensorium не працює
+  - сенсоріум крешить
+replaces_orchestrator: true
+---
+
+# Sensorium Self-Repair Skill
+
+You are repairing **Sensorium itself** (this MCP server + its .NET supervisor) — a live production system the operator depends on. Move deliberately: **establish ground truth from logs and processes before changing anything**, distinguish real crashes from false-death loops, fix the root cause, validate, then ship. Do NOT guess. Do NOT restart blindly.
+
+## Golden rules (read first)
+
+- **Root cause over symptoms.** A restart "fixes" nothing if the bug recurs. Find the exact file:line and the commit that introduced it.
+- **Verify file contents via terminal, never trust cached reads.** The `read_file` tool can serve a STALE copy (it served an old `package.json` version during a real incident). For anything you will edit or reason about critically, confirm with `git show HEAD:<path>` or `Get-Content <path>`.
+- **Never trust a subagent's described diff.** Subagents have hallucinated multi-hundred-line diffs for files that were clean. Always confirm with `git diff` / `git status` in a terminal yourself.
+- **Watch for concurrent autonomous agents.** A Sensorium worker may be committing to this repo *while you work*. Re-check `git HEAD`, branch, and `git log` before and after edits; a fix you were about to write may already be committed.
+- **Git safety:** stage explicit paths only. NEVER `git add -A/./-u`, `git commit -a`. NEVER commit `tmp_*`, dumps, screenshots, or binaries.
+- **Deploy ≠ push.** Pushing to git does not update the running system (see "Ship & deploy").
+
+## Key facts & locations
+
+- **Repo:** the workspace root (e.g. `c:\src\remote-copilot-mcp`). **Runtime data dir:** `~/.remote-copilot-mcp` (`$HOME`/`%USERPROFILE%`).
+- **Logs** under `~/.remote-copilot-mcp/logs/`:
+  - `mcp/server.log` — MCP app log (rotated ~5 MB).
+  - `mcp/mcp-stderr-YYYYMMDD.log` — captured MCP stderr (fatal errors, startup banners, `[wait]`/`[telegram]` activity).
+  - `supervisor/supervisor-YYYYMMDD.log` — supervisor restart/health decisions (very noisy with Telegram `getUpdates`; filter it out).
+- **Process model:** the .NET supervisor process `sensorium-supervisor` supervises a Node MCP server (**HTTP port 3847**) + watcher (**3848**). MCP start command is either `node dist/index.js` (local build) or `npx -y sensorium-mcp@latest` (published). Workers are `claude`/`codex`/`copilot` processes whose command line references `.remote-copilot-mcp`.
+- **`~/.remote-copilot-mcp/server.pid`** holds `{"pid":N}` — should match the port-3847 owner. A mismatch is the signature of a false-death loop.
+- **Windows: `git` is often NOT on PATH.** Locate it (`%LOCALAPPDATA%\Programs\Git\cmd\git.exe` or `C:\Program Files\Git\cmd\git.exe`) and call it by full path.
+- **Prior root causes are logged in memory:** read `/memories/repo/critical-fixes-log.md`, `/memories/repo/keeper-stuck-detection-bug.md`, `/memories/repo/sensorium-architecture.md` before theorizing — the bug may be known.
+
+## Triage workflow
+
+1. **Ground truth first.** `git rev-parse --abbrev-ref HEAD`, `git log --oneline -20`, `git status --short`. Correlate the symptom's start time with recent commits (a bad push is the usual cause).
+2. **Real crash vs. false-death loop.** In `supervisor-YYYYMMDD.log`, count `MCP server process is dead — restarting` events and their cadence. Then in `mcp-stderr-YYYYMMDD.log` check whether ONE process kept serving across those times (continuous `[wait]`/`[telegram]` activity, message-ids incrementing, **no startup banner between restarts**). Continuous serving ⇒ the server never actually died ⇒ **false-death loop** (see failure modes), not a code crash.
+3. **Runtime state.** Enumerate node processes + port owners; compare `server.pid` to the port-3847 owner:
+   ```powershell
+   Get-Process node,sensorium-supervisor -EA SilentlyContinue | Select Id,ProcessName,StartTime
+   Get-NetTCPConnection -LocalPort 3847,3848 -State Listen -EA SilentlyContinue
+   Get-Content ~/.remote-copilot-mcp/server.pid
+   ```
+   `server.pid` ≠ port owner (but port owner alive) confirms a stale-pid false-death loop.
+4. **Find the real fatal error.** In `mcp-stderr-YYYYMMDD.log` search the crash window for `uncaughtException|unhandledRejection|Error:|EADDRINUSE|ECONNREFUSED|Cannot find module|out of memory`. In `supervisor` log, filter noise: `Get-Content <log> | Where-Object { $_ -notmatch 'getUpdates|api.telegram|processing HTTP|response headers' }`.
+
+## Known failure modes
+
+- **False-death restart loop (supervisor).** Health loop declared the MCP dead from a stale `server.pid` without an HTTP cross-check, then spawned a duplicate that couldn't bind 3847 → died → `server.pid` repointed to the dead duplicate → loop every health interval, while the real server kept serving. Fix pattern: probe HTTP (`IsServerReadyAsync`) before killing — restart only when PID dead **and** port silent (`SupervisorWorker.cs`).
+- **Cascade restart after MCP restart.** In-memory `spawnedThreads` empties → `isThreadRunning()` false for all → keeper restarts everything. Fix: rebuild tracking from PID files on startup.
+- **Keeper "stuck" kills keepAlive threads.** `wait_for_instructions` long-poll didn't refresh the per-thread heartbeat → keeper declared stuck → killed. Fix: write `writeThreadHeartbeat()` in the poll loop.
+- **Duplicate thread spawns.** Keeper checks the root threadId but `start_thread` creates a worker with a different id → always "not running" → re-spawns. Fix: track the worker id from the `start_thread` response.
+- **Copilot process dies / no memory.** `copilot.bat` (VS Code wrapper) spawned detached dies silently — prefer `copilot.exe`; and a missing `thread_id` column crashed bootstrap. See critical-fixes-log.
+- **Self-update port-handoff race.** New node self-registers its pid asynchronously, so `server.pid` transiently names a dead pid while HTTP is healthy — the trigger for the false-death loop above.
+
+## Emergency stop & restart
+
+Kill in this order so nothing auto-respawns; be **targeted** (never blanket-kill `node`/`claude`/`copilot` — the operator has an interactive coding session):
+
+1. **Supervisor first** (`sensorium-supervisor`) — otherwise it restarts the MCP you kill.
+2. **MCP server + watcher** — the Node processes owning ports 3847/3848 or whose command line matches `sensorium-mcp` / `remote-copilot-mcp`.
+3. **Worker agents** — only `claude`/`codex`/`copilot` whose command line contains `.remote-copilot-mcp`.
+
+```powershell
+Get-CimInstance Win32_Process |
+  Where-Object { $_.CommandLine -match 'remote-copilot-mcp|sensorium-mcp' -or $_.Name -eq 'sensorium-supervisor.exe' } |
+  Select-Object ProcessId,Name,CommandLine   # ENUMERATE and confirm BEFORE killing
+```
+
+Then `Stop-Process -Id <pid> -Force` in the order above. Verify ports 3847/3848 are free afterward. Do NOT delete lock/PID/data files unless the operator asked. If a process is unkillable (`Access denied`), report it — the operator must intervene.
+
+## Fix, validate, ship
+
+- **Validate before pushing:**
+  - TypeScript: `npx tsc --noEmit` (must exit 0).
+  - Supervisor: `dotnet test supervisor-dotnet/tests/Sensorium.Supervisor.Tests.csproj`.
+  - Node tests: `npm test` — **requires Node ≥ 21** (`node --test` glob). CI matrix runs Node 22/24.
+- **Expert review** the change (dispatch a review subagent; verify security: command injection, path traversal, secrets).
+- **Ship:** `npm version patch --no-git-tag-version` → commit explicit paths (`fix(...)` then `chore(release): x.y.z`) → `git push origin main`.
+- **CI:** confirm green — `gh run watch <run-id> --repo <owner>/<repo> --exit-status`. Fix and re-push until green.
+- **Deploy at runtime (push alone does NOT deploy):**
+  - **MCP server** running via `npx sensorium-mcp@latest` picks up a new version only after **`npm publish`** (the watcher auto-updates). A local `node dist/index.js` needs `npm run build`.
+  - **Supervisor** (.NET) needs a **rebuild + reinstall via `Install-Sensorium.ps1`** to apply supervisor code changes — a running supervisor keeps its old binary in memory.
+
+## After an incident
+
+Record the root cause, the fix, the commit hash, and the version in `/memories/repo/critical-fixes-log.md` (append, don't rewrite) so the next repair starts from knowledge, not zero.