agentgui 1.0.540 → 1.0.542

package/CLAUDE.md

@@ -219,6 +219,91 @@ After update/install completes:
 3. UI version display updates to show new version
 4. Status reverts to "Installed" or "Up-to-date" accordingly
 
+## Base64 Image Rendering in File Read Events
+
+### Problem: Images Displaying as Raw Text
+
+When an agent reads an image file, the streaming event may not have `type='file_read'`. It can arrive with any type (or fall through to the default case in the renderer switch). Without the `renderGeneric` fallback, the image data displays as raw base64 text instead of an `<img>` element.
+
+### Event Structure for Image File Reads
+
+Two nested structures are used by different agent versions. Both must be handled:
+
+**Structure A** (nested under `source`):
+```json
+{
+  "type": "<anything>",
+  "path": "/path/to/image.png",
+  "content": {
+    "source": {
+      "type": "base64",
+      "data": "<base64-string>"
+    },
+    "media_type": "image/png"
+  }
+}
+```
+
+**Structure B** (flat inside `content`):
+```json
+{
+  "type": "<anything>",
+  "path": "/path/to/image.png",
+  "content": {
+    "type": "base64",
+    "data": "<base64-string>"
+  }
+}
+```
+
+**Structure C** (content is raw base64 string, no wrapping object):
+```json
+{
+  "type": "<anything>",
+  "path": "/path/to/image.png",
+  "content": "iVBORw0KGgo..."
+}
+```
+Structure C is detected by `detectBase64Image()` which checks magic-byte prefixes (PNG: `iVBORw0KGgo`, JPEG: `/9j/4AAQ`, WebP: `UklGRi`, GIF: `R0lGODlh`).
+
+### Two Rendering Paths in streaming-renderer.js
+
+**Path 1 – Direct dispatch** (`renderEvent` switch statement):
+- `case 'file_read'` routes directly to `renderFileRead(event)`.
+- Handles all three content structures above.
+
+**Path 2 – Generic fallback** (`renderGeneric`):
+- Called for any event type not matched by the switch (the `default` case).
+- First thing it does: check for `event.content?.source?.type === 'base64'` OR `event.content?.type === 'base64'` AND `event.path` present.
+- If true, delegates to `renderFileRead(event)` so the image renders correctly.
+- Without this fallback, any file-read event that arrives with an unrecognised type displays as raw key-value text.
+
+### MIME Type Resolution in renderFileRead
+
+Priority order inside `renderFileRead`:
+1. `event.media_type` field (explicit)
+2. Detected from magic bytes via `detectBase64Image()` → maps `jpeg` → `image/jpeg`, others → `image/<type>`
+3. Falls back to `application/octet-stream` (shows broken image)
+
+Always include `media_type` on the event when possible. If absent, magic-byte detection covers PNG/JPEG/WebP/GIF automatically.
+
+### Debugging Checklist When Images Show as Text
+
+1. `console.log(event)` the raw event object arriving at the renderer — verify `content` structure.
+2. Check `event.type` — if it is not `'file_read'`, the switch default fires `renderGeneric`.
+3. Confirm `renderGeneric` has the base64 fallback guard at the top (search for `content?.source?.type === 'base64'`).
+4. Confirm `renderFileRead` handles both `content.source.data` and `content.data` paths (both exist in the code).
+5. Verify `event.path` is set — the fallback in `renderGeneric` requires `event.path` to delegate correctly.
+6. If `media_type` is missing and content is not PNG/JPEG/WebP/GIF, add it to the event or extend `detectBase64Image` signatures.
+
+### Why Two Attempts Failed Before the Fix
+
+- Attempt 1: Modified only `renderFileRead` but the event had an unrecognised type so the switch never reached `renderFileRead`.
+- Attempt 2: Added fallback in `renderGeneric` but checked only `event.content?.source?.type === 'base64'` — missed Structure B where data sits directly on `event.content` (no `source` nesting).
+- Fix: `renderGeneric` now checks both structures before falling through to generic key-value rendering.
+
+---
+
 ## Tool Update Testing & Diagnostics
 
 A comprehensive diagnostic page is available at `http://localhost:3000/gm/tool-update-test.html` (`static/tool-update-test.html`) with 7 interactive test sections:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agentgui",
-  "version": "1.0.540",
+  "version": "1.0.542",
   "description": "Multi-agent ACP client with real-time communication",
   "type": "module",
   "main": "server.js",

package/static/js/streaming-renderer.js

@@ -1671,12 +1671,26 @@ class StreamingRenderer {
       let html = '';
       if (event.path) html += this.renderFilePath(event.path);
       if (event.content) {
-        const imageInfo = this.detectBase64Image(event.content);
-        if (imageInfo) {
-          const mimeType = imageInfo.type === 'jpeg' ? 'image/jpeg' : `image/${imageInfo.type}`;
-          html += `<div style="padding:0.5rem;display:flex;flex-direction:column;gap:0.5rem"><img src="data:${mimeType};base64,${this.escapeHtml(imageInfo.data)}" style="max-width:100%;max-height:600px;border-radius:0.375rem;border:1px solid #334155" loading="lazy"><div style="font-size:0.7rem;color:#64748b;font-family:'Monaco','Menlo','Ubuntu Mono',monospace;word-break:break-all">${this.escapeHtml(event.path)}</div></div>`;
+        let base64Data = null;
+        let mimeType = event.media_type || 'application/octet-stream';
+        if (typeof event.content === 'string') {
+          const imageInfo = this.detectBase64Image(event.content);
+          if (imageInfo) {
+            base64Data = imageInfo.data;
+            mimeType = imageInfo.type === 'jpeg' ? 'image/jpeg' : `image/${imageInfo.type}`;
+          }
+        } else if (typeof event.content === 'object' && event.content !== null) {
+          if (event.content.source?.type === 'base64' && event.content.source?.data) {
+            base64Data = event.content.source.data;
+          } else if (event.content.type === 'base64' && event.content.data) {
+            base64Data = event.content.data;
+          }
+        }
+        if (base64Data) {
+          html += `<div style="padding:0.5rem;display:flex;flex-direction:column;gap:0.5rem"><img src="data:${mimeType};base64,${this.escapeHtml(base64Data)}" style="max-width:100%;max-height:600px;border-radius:0.375rem;border:1px solid #334155" loading="lazy"><div style="font-size:0.7rem;color:#64748b;font-family:'Monaco','Menlo','Ubuntu Mono',monospace;word-break:break-all">${this.escapeHtml(event.path)}</div></div>`;
         } else {
-          html += `<pre style="background:#1e293b;padding:0.75rem;border-radius:0.375rem;overflow-x:auto;font-family:'Monaco','Menlo','Ubuntu Mono',monospace;font-size:0.75rem;line-height:1.5;color:#e2e8f0;margin:0.5rem 0 0 0"><code class="lazy-hl">${this.escapeHtml(this.truncateContent(event.content, 2000))}</code></pre>`;
+          const contentStr = typeof event.content === 'string' ? event.content : JSON.stringify(event.content, null, 2);
+          html += `<pre style="background:#1e293b;padding:0.75rem;border-radius:0.375rem;overflow-x:auto;font-family:'Monaco','Menlo','Ubuntu Mono',monospace;font-size:0.75rem;line-height:1.5;color:#e2e8f0;margin:0.5rem 0 0 0"><code class="lazy-hl">${this.escapeHtml(this.truncateContent(contentStr, 2000))}</code></pre>`;
         }
       }
       body.innerHTML = html;
@@ -1982,6 +1996,11 @@ class StreamingRenderer {
    * Render generic event with formatted key-value pairs
    */
   renderGeneric(event) {
+    // Check if this is actually a file read with base64 image content
+    if ((event.content?.source?.type === 'base64' || event.content?.type === 'base64') && event.path) {
+      return this.renderFileRead(event);
+    }
+
     const div = document.createElement('div');
     div.className = 'event-generic mb-3 p-3 bg-gray-100 dark:bg-gray-800 rounded text-sm';
     div.dataset.eventId = event.id || '';