@ducci/jarvis 1.0.28 → 1.0.29

package/docs/findings/012-empty-nudge-loses-recovery-text.md

@@ -0,0 +1,121 @@
+# Finding 012: Empty-Content Nudge Includes Tools and Loses Recovery Text
+
+**Date:** 2026-03-02
+**Severity:** Medium — user sees generic error when model produces a partial recovery response
+**Status:** Fixed
+
+---
+
+## Observed Session
+
+Session `21fb43a7-2b11-4208-99fb-e6b54fddc07b`, entry 9 in session.jsonl:
+
+```
+status=format_error
+model=nvidia/nemotron-3-nano-30b-a3b:free
+iteration=3
+userInput='Ok. Read the results folder. Is there anything?'
+logSummary='Model returned non-JSON final response after recovery attempts.'
+response='The model did not produce a response. Please try again.'
+```
+
+The user received: **"The model did not produce a response. Please try again."**
+
+---
+
+## What Happened
+
+1. The agent executed two tool calls:
+   - `list_dir /root/.jarvis/projects/cybersecurity/results` → success
+   - `exec "list_dir /root/.jarvis/projects/cybersecurity/results/dviet.de"` → exit 127 (`list_dir: not found`)
+     - The model confused the `list_dir` jarvis tool with a shell command
+
+2. After the failed exec, the model returned `assistantMessage.content = null` with no `tool_calls` — it "went silent"
+
+3. Finding 011's empty-content nudge was triggered
+
+4. The nudge **also failed** — no valid JSON response was produced
+
+5. The agent fell through to `format_error` with the fallback message
+
+---
+
+## Bug Chain
+
+### Bug 1 — toolDefs included in empty nudge
+
+```js
+const nudgeResult = await callModelWithFallback(client, config, emptyNudge, toolDefs);
+```
+
+When the model is confused after a tool failure, it may respond to the nudge with **another tool call** instead of text. If it does:
+
+```
+nudgeResult.choices[0].message.content = null
+nudgeContent = ''
+JSON.parse('') → throws
+catch: // Give up — content stays ''
+```
+
+The model had an opportunity to call more tools instead of producing a text response — the wrong behavior for a recovery nudge.
+
+### Bug 2 — content assigned after parse
+
+```js
+const nudgeContent = nudgeResult.choices[0]?.message?.content || '';
+parsed = JSON.parse(nudgeContent);   // ← throws on non-JSON or empty
+content = nudgeContent;              // ← only reached if parse succeeded
+```
+
+If the model responds to the nudge with non-empty but non-JSON text (e.g. a plain English answer), `JSON.parse` throws and `content` is **never updated**. The non-JSON text is discarded. The `!parsed` handler then shows the fallback message instead of the model's actual text.
+
+---
+
+## Difference from Finding 011
+
+| Finding | Problem | Trigger |
+|---------|---------|---------|
+| 011 | Empty model response propagates to Telegram | Initial empty content, no recovery chain |
+| 012 | Recovery nudge discards best-effort text; model can respond with tool call | Recovery nudge called with toolDefs + content assigned after parse |
+
+Finding 012 is a refinement of the recovery path introduced in Finding 011.
+
+---
+
+## Fix
+
+### `src/server/agent.js` — empty-content nudge block
+
+**Before:**
+```js
+const nudgeResult = await callModelWithFallback(client, config, emptyNudge, toolDefs);
+const nudgeContent = nudgeResult.choices[0]?.message?.content || '';
+parsed = JSON.parse(nudgeContent);
+content = nudgeContent;
+```
+
+**After:**
+```js
+// No tools: force text response, prevent model from calling another tool
+const nudgeResult = await callModelWithFallback(client, config, emptyNudge, []);
+const nudgeContent = nudgeResult.choices[0]?.message?.content || '';
+// Persist before parsing — if JSON parse throws, content still carries the
+// model's best-effort text so the !parsed handler can show it to the user
+if (nudgeContent.trim()) {
+  content = nudgeContent;
+}
+parsed = JSON.parse(nudgeContent);
+```
+
+---
+
+## Outcome
+
+| Nudge response | Before | After |
+|---|---|---|
+| Valid JSON | Clean recovery | Clean recovery (no change) |
+| Non-JSON text | Text discarded, fallback shown | Text shown to user |
+| Tool call (no content) | content='', fallback shown | Less likely; content='', fallback shown |
+| Empty again | content='', fallback shown | content='', fallback shown (no change) |
+
+The user in the observed session would have received the model's best-effort text about the results folder contents, rather than "The model did not produce a response. Please try again."

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ducci/jarvis",
-  "version": "1.0.28",
+  "version": "1.0.29",
   "description": "A fully automated agent system that lives on a server.",
   "main": "./src/index.js",
   "type": "module",

package/src/server/agent.js

@@ -218,17 +218,24 @@ async function runAgentLoop(client, config, session, prepareMessages) {
     if (!content.trim()) {
       // Model returned no content at all — use a targeted nudge instead of the
       // standard JSON recovery chain (designed for non-empty non-JSON responses).
+      // Send with no tools so the model cannot respond with another tool call,
+      // which would leave content empty and discard any recovery text.
       try {
         const emptyNudge = [
           ...preparedMessages,
           { role: 'user', content: 'You returned an empty response. ' + FORMAT_NUDGE },
         ];
-        const nudgeResult = await callModelWithFallback(client, config, emptyNudge, toolDefs);
+        const nudgeResult = await callModelWithFallback(client, config, emptyNudge, []);
         const nudgeContent = nudgeResult.choices[0]?.message?.content || '';
+        // Persist nudge text before parsing — if JSON parse throws, content still
+        // carries the model's best-effort text so the !parsed handler can show it
+        // rather than falling back to "The model did not produce a response."
+        if (nudgeContent.trim()) {
+          content = nudgeContent;
+        }
         parsed = JSON.parse(nudgeContent);
-        content = nudgeContent;
       } catch {
-        // Give up — fall through to !parsed handler below
+        // Fall through to !parsed handler; content may now carry the nudge text
       }
     } else {
       try {