nexo-brain 7.11.1 → 7.11.3

package/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "nexo-brain",
-  "version": "7.11.1",
+  "version": "7.11.3",
   "description": "Local cognitive runtime for Claude Code \u2014 persistent memory, overnight learning, doctor diagnostics, personal scripts, recovery-aware jobs, startup preflight, and optional dashboard/power helper.",
   "author": {
     "name": "NEXO Brain",

package/README.md

@@ -18,7 +18,11 @@
 
 [Watch the overview video](https://nexo-brain.com/watch/) · [Watch on YouTube](https://www.youtube.com/watch?v=i2lkGhKyVqI) · [Open the infographic](https://nexo-brain.com/assets/nexo-brain-infographic-v5.png)
 
-Version `7.11.1` is the current packaged-runtime line. Patch release — caches the runtime fingerprint by `(file_count, size_total, max_mtime)` signature so MCP startup and the per-tool-call `resolve_restart_required` skip the 263-file rehash when nothing on disk changed. ~11× speedup warm path (~40ms → ~3.7ms locally), ~10-20s/day saved across Claude Code / Codex / headless / deep-sleep / cron startups. Cache miss is always safe (falls through to full hash and self-repairs). Default `use_cache=False` keeps `plugins/update.py` on the ground-truth path around `git pull` / `npm update`. Builds on the v7.11.0 runtime fingerprint that gates `mcp-restart-required.json`. Full write-up in [`docs/runtime-fingerprint.md`](docs/runtime-fingerprint.md).
+Version `7.11.3` is the current packaged-runtime line. Patch release — root-cause fix for the `mcp_restart_required` lockup that v7.11.2 only masked at the enforcer layer. `_FINGERPRINT_EXCLUDE_DIRS` in `src/runtime_versioning.py` was missing `"versions"`, so `compute_mcp_runtime_fingerprint()` walked into `core/versions/<old>/**.py` whenever it was called against the live runtime root. `installed_runtime_fingerprint()` (which resolves through `active_runtime_root()` → `core/versions/<active>/`) returned a clean per-snapshot hash, while `prime_process_fingerprint()` (which starts from `Path(__file__).resolve().parent` → live `core/`) accumulated every retained snapshot. The two never matched after the second-ever `nexo update` on a host. Every update wrote `mcp-restart-required.json` and the marker could never be cleared by `_ack_current_client_if_restarted()` because the `installed_fp != process_fp` test always returned `True`. Every non-allowlisted MCP tool (`nexo_reminders`, `nexo_smart_startup`, `nexo_guard_check`, `nexo_task_open`, …) returned `{"error": "mcp_restart_required", "reason": "fingerprint_mismatch"}` indefinitely, even after the operator restarted the client (the new client connected to the same server with the same cached `PROCESS_FINGERPRINT`). v7.11.2 silenced the enforcer-side noise but the marker itself stayed stuck, so user-driven calls kept failing across sessions. Adding `"versions"` to `_FINGERPRINT_EXCLUDE_DIRS` restores parity: both fingerprint computations now hash the same set of files regardless of which entry path the caller starts from. 1 new regression test (`test_fingerprint_ignores_versions_subtree`) seeds three snapshot directories under `versions/` and asserts the fingerprint does not shift. The two existing exclude-dir tests now also cover `"versions"`. All 21 tests in `tests/test_runtime_fingerprint.py` stay green.
+
+Previously in `7.11.2`: patch release — two reliability fixes in the same family ("components ignoring signals they should respect"): (1) `STUCK CRON REAPER` added to `nexo-watchdog.sh` and (2) the Guardian/Enforcer now honors the `mcp-restart-required` marker. The watchdog reaper closes the v5.8.1 in-flight gap: truly hung wrappers (e.g. headless `claude --bare` blocked on an MCP that flagged `mcp_restart_required`) used to hold their slot for days. The reaper sweeps `cron_runs` rows with `ended_at IS NULL` past `stuck_after_seconds` (per-cron from `manifest.json`, fallback 12h global), SIGTERMs the wrapper (trap closes row at `exit 143`), grace 10s, SIGKILL on survivors. Generous defaults (deep-sleep 8h, sleep/evolution 4h) prevent any v5.8.1 regression. The enforcer gate skips `nexo_*`-mentioning reminders when the marker file is present (cached per-instance, 30s TTL); reminders that don't reference `nexo_*` still fire. 12 new tests; 3 existing watchdog tests + 52 existing enforcer tests stay green.
+
+Previously in `7.11.1`: patch release — caches the runtime fingerprint by `(file_count, size_total, max_mtime)` signature so MCP startup and the per-tool-call `resolve_restart_required` skip the 263-file rehash when nothing on disk changed. ~11× speedup warm path (~40ms → ~3.7ms locally), ~10-20s/day saved across Claude Code / Codex / headless / deep-sleep / cron startups. Cache miss is always safe (falls through to full hash and self-repairs). Default `use_cache=False` keeps `plugins/update.py` on the ground-truth path around `git pull` / `npm update`. Builds on the v7.11.0 runtime fingerprint that gates `mcp-restart-required.json`. Full write-up in [`docs/runtime-fingerprint.md`](docs/runtime-fingerprint.md).
 
 Previously in `7.10.0`: minor release — **removes the LLM proxy override path that 7.9.28 → 7.9.34 introduced**. Background: 7.9.28 added two opt-in files at `~/.nexo/config/llm_endpoint.json` and `~/.nexo/config/auth_provider.json` that let a third-party orchestrator (NEXO Desktop) redirect every Anthropic SDK call from Brain to a custom proxy and resolve the bearer via a local helper, with concrete model names translated to wire aliases (`nexo-max`, `nexo-high`, `nexo-medium`, `nexo-low`, `nexo-mini`) and an `Idempotency-Key` per request. NEXO Desktop's commercial model has changed: Desktop is now a wrapper over the user's own Claude Code subscription (Max / Pro), with a separate Desktop licence. Brain calls go directly to `api.anthropic.com` using the user's existing OAuth (the one stored under `~/.claude/` and consumed by Claude Code spawns) or a plain `ANTHROPIC_API_KEY`. There is no NEXO bearer, no NEXO proxy, no NEXO credit accounting in this codebase. Every proxy symbol is gone from `call_model_raw.py` and `agent_runner.py`; the proxy-specific tests and `docs/api/override-files.md` are removed; any pre-existing override files on disk are simply ignored from this release forward.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "nexo-brain",
-  "version": "7.11.1",
+  "version": "7.11.3",
   "mcpName": "io.github.wazionapps/nexo",
   "description": "NEXO Brain — Shared brain for AI agents. Persistent memory, semantic RAG, natural forgetting, metacognitive guard, trust scoring, 150+ MCP tools. Works with Claude Code, Codex, Claude Desktop & any MCP client. 100% local, free.",
   "homepage": "https://nexo-brain.com",

package/src/crons/manifest.json

@@ -13,6 +13,7 @@
       "recovery_policy": "catchup",
       "idempotent": true,
       "max_catchup_age": 172800,
+      "stuck_after_seconds": 28800,
       "run_on_boot": true,
       "run_on_wake": true
     },
@@ -38,6 +39,7 @@
       "recovery_policy": "catchup",
       "idempotent": true,
       "max_catchup_age": 172800,
+      "stuck_after_seconds": 14400,
       "run_on_boot": true,
       "run_on_wake": true
     },
@@ -140,6 +142,7 @@
       "recovery_policy": "catchup",
       "idempotent": true,
       "max_catchup_age": 1209600,
+      "stuck_after_seconds": 14400,
       "run_on_boot": true,
       "run_on_wake": true
     },
@@ -295,6 +298,7 @@
       "recovery_policy": "run_once_on_wake",
       "idempotent": true,
       "max_catchup_age": 1200,
+      "stuck_after_seconds": 600,
       "run_on_boot": false,
       "run_on_wake": true
     },
@@ -308,6 +312,7 @@
       "recovery_policy": "run_once_on_wake",
       "idempotent": true,
       "max_catchup_age": 7200,
+      "stuck_after_seconds": 1800,
       "run_on_boot": false,
       "run_on_wake": true
     },
@@ -321,6 +326,7 @@
       "recovery_policy": "catchup",
       "idempotent": true,
       "max_catchup_age": 86400,
+      "stuck_after_seconds": 1800,
       "run_on_boot": false,
       "run_on_wake": true
     }

package/src/enforcement_engine.py

@@ -2520,6 +2520,44 @@ class HeadlessEnforcer:
     # the per-rule tag collision check, and time-dedup at the call site.
     _LEGACY_TAG_PREFIXES = ("after:", "periodic_msgs:", "periodic_time:", "start:")
 
+    @staticmethod
+    def _mcp_restart_marker_path() -> "Path":
+        """Resolve the path to the MCP restart-required marker on disk.
+
+        The marker is written by `plugins/update.py` when a `nexo update`
+        actually changes runtime `.py` bytes (cf. v7.11.0 fingerprint
+        gating). Honors the F0.6 runtime/operations/ canonical layout
+        with a fall-back to the pre-F0.6 operations/ legacy layout so
+        half-migrated installs are still detected correctly.
+        """
+        from pathlib import Path as _Path
+        home = _Path(os.environ.get("NEXO_HOME", str(_Path.home() / ".nexo")))
+        new = home / "runtime" / "operations" / "mcp-restart-required.json"
+        if new.is_file():
+            return new
+        legacy = home / "operations" / "mcp-restart-required.json"
+        return legacy if legacy.is_file() else new
+
+    def _mcp_restart_pending(self) -> bool:
+        """Return True if the MCP server has a restart-required marker on disk.
+
+        Cached per-instance with a 30s TTL: the marker rarely changes mid-
+        session (it's written by `nexo update` and cleared by the next
+        client restart) but a TTL keeps long-lived enforcer instances from
+        getting stuck on a stale negative cache if the operator runs
+        `nexo update` mid-session without restarting.
+        """
+        cached_at = getattr(self, "_mcp_restart_pending_cache_at", 0.0)
+        if (time.time() - cached_at) < 30.0:
+            return getattr(self, "_mcp_restart_pending_cache", False)
+        try:
+            result = self._mcp_restart_marker_path().is_file()
+        except Exception:  # noqa: BLE001 — never block enforcement on path errors
+            result = False
+        self._mcp_restart_pending_cache = result
+        self._mcp_restart_pending_cache_at = time.time()
+        return result
+
     def _enqueue(self, prompt: str, tag: str, rule_id: str = ""):
         """Enqueue an injection. Mirrors Desktop _enqueue for parity.
 
@@ -2535,6 +2573,21 @@ class HeadlessEnforcer:
         """
         if any(q["tag"] == tag for q in self.injection_queue):
             return
+        # v7.11.2: suppress reminders that ask the agent to call nexo_*
+        # tools while the MCP server has a restart-required marker on
+        # disk. Without this gate every periodic ping ("Execute
+        # nexo_session_diary_write", "Execute nexo_smart_startup",
+        # nexo_guard_check pre-Edit, etc) returns mcp_restart_required
+        # and the agent burns cycles on guaranteed no-ops. Reminders that
+        # don't reference nexo_* (R23 deploy guards, R25 nora/maria
+        # read-only, etc) still fire — they don't depend on the MCP.
+        if "nexo_" in prompt and self._mcp_restart_pending():
+            _logger.info(
+                "SKIP: %s — mcp_restart_required marker present (rule_id=%s)",
+                tag,
+                rule_id or "?",
+            )
+            return
         legacy = tag.startswith(self._LEGACY_TAG_PREFIXES)
         if legacy:
             tool = tag.split(":")[-1].split("->")[-1]

package/src/runtime_versioning.py

@@ -33,6 +33,7 @@ _FINGERPRINT_EXCLUDE_DIRS = frozenset({
     "__pycache__",
     "node_modules",
     ".git",
+    "versions",
 })
 RESTART_CLIENT_ACTIONS = {
     "claude_desktop": "restart_client_required",

package/src/scripts/nexo-watchdog.sh

@@ -530,6 +530,183 @@ json_escape() {
   echo "$1" | sed 's/\\/\\\\/g; s/"/\\"/g; s/	/ /g' | tr '\n' ' '
 }
 
+# ============================================================================
+# STUCK CRON REAPER (v7.11.2)
+# ============================================================================
+# Mirror image of the v5.8.1 in-flight detection. The v5.8.1 fix taught the
+# watchdog to leave running jobs alone when their cron_runs row was open
+# (started_at present, ended_at NULL) — that closed the loop where the
+# watchdog kept kickstart -k'ing deep-sleep mid-flight (2026-04-14..17).
+#
+# But the same restraint became the new failure mode: when a wrapper child
+# truly hangs (e.g. headless `claude --bare` blocked on an MCP that flagged
+# `mcp_restart_required`), the row stays open forever, no new tick can run
+# (the next wrapper sees "Another instance running. Skipping"), and the
+# watchdog's only response was WARN. Morning brief, followup runner, and
+# orchestrator-v2 went silent for days because of this.
+#
+# The reaper closes that gap without bringing back the v5.8.1 bug:
+#   * Per-cron threshold via `stuck_after_seconds` in manifest.json.
+#   * Generous default (12h) so legitimate long jobs keep running.
+#   * Override deep-sleep to 8h, sleep/evolution to 4h — well above their
+#     real worst-case so the v5.8.1 incident cannot repeat.
+#   * Reaper sends SIGTERM to the wrapper — its trap (line 187) closes the
+#     cron_runs row exit_code=143 and propagates to the child. Only after
+#     a 10s grace does it escalate to SIGKILL on wrapper + descendants.
+#   * If no wrapper PID is alive (orphan row), the reaper just closes the
+#     row in-band with exit_code=137 so the next tick can run.
+# ============================================================================
+
+STUCK_DEFAULT_SECONDS="${STUCK_DEFAULT_SECONDS:-43200}"  # 12h
+STUCK_KILL_GRACE="${STUCK_KILL_GRACE:-10}"
+TOTAL_REAPED=0
+
+# Skip cron_ids that should never be reaped from inside a watchdog tick.
+# 'watchdog' is us — reaping ourselves would be self-immolation.
+STUCK_REAPER_SKIP="watchdog"
+
+_build_stuck_thresholds_from_manifest() {
+  if [ ! -f "$MANIFEST_FILE" ]; then
+    return
+  fi
+  python3 - "$MANIFEST_FILE" <<'PY' 2>/dev/null
+import json, sys
+try:
+    with open(sys.argv[1]) as f:
+        data = json.load(f)
+except Exception:
+    sys.exit(0)
+for c in data.get('crons', []):
+    cid = c.get('id')
+    th = c.get('stuck_after_seconds')
+    if cid and isinstance(th, (int, float)) and th > 0:
+        print(f"{cid}|{int(th)}")
+PY
+}
+
+STUCK_THRESHOLDS_RAW=""
+_load_stuck_thresholds() {
+  STUCK_THRESHOLDS_RAW=$(_build_stuck_thresholds_from_manifest)
+}
+
+lookup_stuck_threshold() {
+  local cron_id="$1"
+  if [ -z "$STUCK_THRESHOLDS_RAW" ]; then
+    echo "$STUCK_DEFAULT_SECONDS"
+    return
+  fi
+  local line
+  line=$(echo "$STUCK_THRESHOLDS_RAW" | grep "^${cron_id}|" | head -1)
+  if [ -n "$line" ]; then
+    echo "$line" | cut -d'|' -f2
+  else
+    echo "$STUCK_DEFAULT_SECONDS"
+  fi
+}
+
+find_wrapper_pids() {
+  local cron_id="$1"
+  # Match the wrapper's exact arg slot: "nexo-cron-wrapper.sh CRON_ID "
+  # The trailing space prevents prefix collisions (e.g. "morning-agent" vs
+  # a hypothetical "morning-agent-v2").
+  pgrep -f "nexo-cron-wrapper\.sh ${cron_id} " 2>/dev/null
+}
+
+reap_stuck_cron_pids() {
+  local cron_id="$1"
+  local pids
+  pids=$(find_wrapper_pids "$cron_id")
+  if [ -z "$pids" ]; then
+    # No wrapper alive — caller should fall through to in-band row cleanup.
+    return 1
+  fi
+  log_repair "STUCK REAPER: SIGTERM to wrapper PIDs ($cron_id): $(echo "$pids" | tr '\n' ' ')"
+  for pid in $pids; do
+    kill -TERM "$pid" 2>/dev/null || true
+  done
+  # Grace period — the wrapper trap (TERM → forward to child → finalize_row)
+  # needs a few seconds to close the cron_runs row cleanly.
+  local waited=0
+  local still
+  while [ $waited -lt "$STUCK_KILL_GRACE" ]; do
+    sleep 1
+    waited=$((waited + 1))
+    still=$(find_wrapper_pids "$cron_id")
+    [ -z "$still" ] && break
+  done
+  # Escalate to SIGKILL for any survivor (wrapper + descendants).
+  local survivors
+  survivors=$(find_wrapper_pids "$cron_id")
+  if [ -n "$survivors" ]; then
+    log_repair "STUCK REAPER: SIGKILL escalation ($cron_id): $(echo "$survivors" | tr '\n' ' ')"
+    for pid in $survivors; do
+      # Kill descendants first so they don't get reparented to PID 1.
+      pkill -KILL -P "$pid" 2>/dev/null || true
+      kill -KILL "$pid" 2>/dev/null || true
+    done
+    sleep 1
+  fi
+  # Last sanity check.
+  if [ -n "$(find_wrapper_pids "$cron_id")" ]; then
+    log "STUCK REAPER: failed to kill wrapper for $cron_id (still alive after SIGKILL)"
+    return 2
+  fi
+  return 0
+}
+
+finalize_stuck_db_row() {
+  local row_id="$1"
+  local cron_id="$2"
+  [ ! -f "$DB_PATH" ] && return 1
+  sqlite3 "$DB_PATH" "
+    UPDATE cron_runs
+       SET ended_at = strftime('%Y-%m-%d %H:%M:%S','now'),
+           exit_code = 137,
+           summary = 'stuck row reaped by watchdog: wrapper PID gone',
+           error = 'Watchdog STUCK REAPER: orphan in-flight row cleaned up',
+           duration_secs = CAST(strftime('%s','now') - strftime('%s', started_at) AS REAL)
+     WHERE id = $row_id;
+  " 2>/dev/null
+  log_repair "STUCK REAPER: cleaned up zombie cron_runs row id=$row_id ($cron_id)"
+}
+
+run_stuck_reaper() {
+  [ ! -f "$DB_PATH" ] && return 0
+  _load_stuck_thresholds
+  local row_id cron_id age_secs threshold
+  while IFS='|' read -r row_id cron_id age_secs; do
+    [ -z "$row_id" ] && continue
+    [ -z "$cron_id" ] && continue
+    # Skip self and any explicitly-protected cron_ids.
+    case " $STUCK_REAPER_SKIP " in
+      *" $cron_id "*) continue ;;
+    esac
+    threshold=$(lookup_stuck_threshold "$cron_id")
+    if [ "$age_secs" -gt "$threshold" ]; then
+      log "STUCK REAPER: cron_id=$cron_id row_id=$row_id age=${age_secs}s threshold=${threshold}s — reaping"
+      if reap_stuck_cron_pids "$cron_id"; then
+        # Wrapper trap closes the row with exit 143; nothing else to do.
+        TOTAL_REAPED=$((TOTAL_REAPED + 1))
+      else
+        # No wrapper alive (orphan zombie row) — close it in-band so the
+        # next tick of this cron isn't blocked by "Another instance running".
+        finalize_stuck_db_row "$row_id" "$cron_id"
+        TOTAL_REAPED=$((TOTAL_REAPED + 1))
+      fi
+    fi
+  done < <(sqlite3 -separator '|' "$DB_PATH" "
+    SELECT id, cron_id, CAST(strftime('%s','now') - strftime('%s', started_at) AS INTEGER)
+      FROM cron_runs
+     WHERE ended_at IS NULL
+     ORDER BY id DESC;
+  " 2>/dev/null)
+  if [ "$TOTAL_REAPED" -gt 0 ]; then
+    log "STUCK REAPER: complete — reaped $TOTAL_REAPED stuck cron(s)"
+  fi
+}
+
+run_stuck_reaper
+
 # ============================================================================
 # RUN CHECKS
 # ============================================================================
@@ -1023,6 +1200,7 @@ cat > "$STATUS_JSON" <<JSONEOF
     "warn": $TOTAL_WARN,
     "fail": $TOTAL_FAIL,
     "healed": $TOTAL_HEALED,
+    "reaped": $TOTAL_REAPED,
     "overall": "$OVERALL"
   },
   "launch_agents": [
@@ -1047,7 +1225,7 @@ cat > "$REPORT_TXT" <<REPORTEOF
 ======================================================
   NEXO WATCHDOG REPORT — $TS
 ======================================================
-  PASS: $TOTAL_PASS  |  HEALED: $TOTAL_HEALED  |  WARN: $TOTAL_WARN  |  FAIL: $TOTAL_FAIL  |  TOTAL: $TOTAL
+  PASS: $TOTAL_PASS  |  HEALED: $TOTAL_HEALED  |  WARN: $TOTAL_WARN  |  FAIL: $TOTAL_FAIL  |  REAPED: $TOTAL_REAPED  |  TOTAL: $TOTAL
   OVERALL: $OVERALL
 ======================================================
 
@@ -1261,4 +1439,4 @@ fi
 # ============================================================================
 # LOG SUMMARY
 # ============================================================================
-log "Complete: PASS=$TOTAL_PASS HEALED=$TOTAL_HEALED WARN=$TOTAL_WARN FAIL=$TOTAL_FAIL"
+log "Complete: PASS=$TOTAL_PASS HEALED=$TOTAL_HEALED WARN=$TOTAL_WARN FAIL=$TOTAL_FAIL REAPED=$TOTAL_REAPED"