zubo 0.1.24 → 0.1.26

package/CHANGELOG.md

@@ -0,0 +1,20 @@
+# Changelog
+
+## 0.1.25 - 2026-02-17
+
+- Added `zubo eval` reliability command with deterministic checks for slash commands, memory explainability, and dry-run safety.
+- Added unified slash command write-actions:
+  - `/model set <provider/model>`
+  - `/permissions set <tool> <auto|confirm|deny>`
+  - `/budget pause|resume`
+- Added configurable memory retrieval tuning:
+  - `memoryRetrieval.contextTopK`
+  - `memoryRetrieval.minConfidence`
+- Added configurable runtime tool policy controls:
+  - `toolScopes.allowed`
+  - `toolScopes.dryRunByDefault`
+  - `toolPermissions.<tool>`
+- Updated dashboard settings UI with memory retrieval and tool safety controls, including preset buttons and inline guidance.
+- Improved memory explainability display in dashboard and memory search outputs (match type, confidence, reasons).
+- Updated front-facing docs (`README`, CLI, config, memory docs) for new commands and settings.
+- Added CI gate for `zubo eval`.

package/README.md

@@ -28,7 +28,8 @@
 
 - **11+ LLM providers** — Anthropic, OpenAI, Google Gemini, Ollama, Groq, Together, OpenRouter, DeepSeek, xAI, Fireworks, LM Studio, and any OpenAI-compatible endpoint. Smart routing sends simple queries to fast models automatically.
 - **7 channels** — Telegram, Discord, Slack, WhatsApp, Signal, Email, Web Chat
-- **Persistent memory** — Vector + full-text hybrid search with ONNX embeddings and FTS5. Remembers every conversation, preference, and fact — forever.
+- **Persistent memory** — Vector + full-text hybrid search with ONNX embeddings and FTS5. Remembers every conversation, preference, and fact — forever.
+- **Memory explainability** — Memory matches include confidence and why they were selected (keyword, semantic, or hybrid match).
 - **25+ built-in tools** — Web search (Brave + DuckDuckGo), file ops, code execution, APIs, sub-agent delegation, knowledge graph, memory pruning, reminders, and automatic failover between providers.
 - **Extensible skills** — Build custom skills in TypeScript. Share them on the registry. Install community skills with one command.
 - **9 integrations** — GitHub, Google (Gmail, Calendar, Docs, Drive, Sheets), Notion, Linear, Jira, Slack, Twitter + Claude Code and MCP
@@ -36,7 +37,8 @@
 - **Natural language scheduling** — "Every weekday at 9am" just works. Cron jobs, heartbeat, proactive tasks.
 - **Voice** — Speech-to-text (Whisper, local whisper.cpp), text-to-speech (OpenAI, ElevenLabs), and continuous voice conversation mode
 - **Personal tools** — Todos, notes, preferences, topics, and follow-ups — all manageable from the dashboard or via chat
-- **Dashboard** — Built-in web UI with analytics, memory management, Ollama model manager, personal tools, and settings
+- **Dashboard** — Built-in web UI with analytics, memory management, Ollama model manager, personal tools, and settings
+- **Safety controls** — Tool scope allowlists and dry-run-by-default mode for risky tools, configurable in the dashboard
 - **Document ingestion** — Upload PDF, DOCX, XLSX, PPTX, TXT, CSV, JSON, and more
 - **Budget controls** — Daily/monthly spending limits with per-model cost tracking
 - **100% local** — SQLite database, local vector store. Your data never leaves your machine.
@@ -128,12 +130,28 @@ zubo model [provider/model] Show or switch LLM
 zubo skills                Manage skills
 zubo install <name>        Install from registry
 zubo search <query>        Search the registry
-zubo voice                 Continuous voice conversation mode
-zubo auth create-key       Create an API key
-zubo export / import       Backup and restore
+zubo voice                 Continuous voice conversation mode
+zubo eval                  Run reliability + safety checks
+zubo auth create-key       Create an API key
+zubo export / import       Backup and restore
 ```
 
-Full reference at [zubo.bot/docs/cli.html](https://zubo.bot/docs/cli.html).
+Full reference at [zubo.bot/docs/cli.html](https://zubo.bot/docs/cli.html).
+
+## Unified Slash Commands
+
+Across WebChat, Telegram, Discord, Slack, and other channels:
+
+- `/help` — list available commands
+- `/status` — runtime status
+- `/memory <query>` — search saved memory with confidence metadata
+- `/model` — show current provider/model
+- `/model set <provider/model>` — switch active model at runtime
+- `/tools [filter]` — list available tools
+- `/permissions <tool>` — view tool permission + scopes
+- `/permissions set <tool> <auto|confirm|deny>` — override tool permission
+- `/budget` — view budget usage and limits
+- `/budget pause|resume` — pause/resume budget enforcement
 
 ## Contributing
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zubo",
-  "version": "0.1.24",
+  "version": "0.1.26",
   "description": "Your AI agent that never forgets. Persistent memory, 25+ tools, 7 channels, 11+ LLM providers — runs entirely on your machine.",
   "license": "MIT",
   "author": "thomaskanze",
@@ -32,6 +32,7 @@
     "logs": "bun run src/index.ts logs",
     "logs:follow": "bun run src/index.ts logs --follow",
     "model": "bun run src/index.ts model",
+    "eval": "bun run src/index.ts eval",
     "skills": "bun run src/index.ts skills",
     "dev": "bun run --watch src/index.ts start",
     "desktop:dev": "cd desktop && npm run dev",

package/src/channels/dashboard.html.ts

@@ -1525,8 +1525,8 @@ export const DASHBOARD_HTML = `<!DOCTYPE html>
             </div>
           </div>
 
-          <div class="settings-section">
-            <h3 class="settings-title" data-tooltip="How often Zubo checks for tasks">Background Check Interval</h3>
+          <div class="settings-section">
+            <h3 class="settings-title" data-tooltip="How often Zubo checks for tasks">Background Check Interval</h3>
             <p class="settings-desc">How often Zubo checks for reminders, scheduled tasks, and updates. Default: every 30 minutes.</p>
             <div class="settings-grid">
               <div class="settings-field">
@@ -1537,11 +1537,58 @@ export const DASHBOARD_HTML = `<!DOCTYPE html>
             <div style="margin-top: 16px; display: flex; gap: 10px; align-items: center;">
               <button class="btn btn-primary" onclick="saveHeartbeat()">Save</button>
               <span id="heartbeat-status" class="status-text"></span>
-            </div>
-          </div>
-
-          <div class="settings-section">
-            <h3 class="settings-title">Configuration</h3>
+            </div>
+          </div>
+
+          <div class="settings-section">
+            <h3 class="settings-title">Memory Retrieval</h3>
+            <p class="settings-desc">Control how many memory chunks are injected into chat context and the minimum confidence threshold.</p>
+            <div class="settings-grid">
+              <div class="settings-field">
+                <label class="settings-label" for="memory-context-topk">Context Top-K</label>
+                <input id="memory-context-topk" type="number" class="settings-input" min="1" max="10" step="1" placeholder="3">
+              </div>
+              <div class="settings-field">
+                <label class="settings-label" for="memory-min-confidence">Min Confidence (0-1)</label>
+                <input id="memory-min-confidence" type="number" class="settings-input" min="0" max="1" step="0.05" placeholder="0">
+              </div>
+            </div>
+            <div style="margin-top: 16px; display: flex; gap: 10px; align-items: center;">
+              <button class="btn btn-primary" onclick="saveMemoryRetrievalSettings()">Save</button>
+              <button class="btn btn-ghost" onclick="applyMemoryPreset('balanced')">Balanced</button>
+              <button class="btn btn-ghost" onclick="applyMemoryPreset('strict')">Strict</button>
+              <span id="memory-retrieval-status" class="status-text"></span>
+            </div>
+            <p class="settings-desc" style="margin-top:10px;margin-bottom:0;">Recommended: <code>Top-K 3-5</code> and <code>min confidence 0.2-0.35</code>.</p>
+          </div>
+
+          <div class="settings-section">
+            <h3 class="settings-title">Tool Safety</h3>
+            <p class="settings-desc">Limit tool scopes and optionally force dry-run mode by default for risky tools.</p>
+            <div class="settings-grid">
+              <div class="settings-field">
+                <label class="settings-label" for="tool-scopes-allowed">Allowed Scopes (comma-separated)</label>
+                <input id="tool-scopes-allowed" type="text" class="settings-input" placeholder="memory,network_read,filesystem_read">
+              </div>
+              <div class="settings-field">
+                <label class="settings-label" for="tool-scopes-dry-run">Dry-Run By Default</label>
+                <select id="tool-scopes-dry-run" class="settings-select">
+                  <option value="false">No</option>
+                  <option value="true">Yes</option>
+                </select>
+              </div>
+            </div>
+            <div style="margin-top: 16px; display: flex; gap: 10px; align-items: center;">
+              <button class="btn btn-primary" onclick="saveToolScopeSettings()">Save</button>
+              <button class="btn btn-ghost" onclick="applyToolScopePreset('safe')">Safe</button>
+              <button class="btn btn-ghost" onclick="applyToolScopePreset('balanced')">Balanced</button>
+              <span id="tool-scopes-status" class="status-text"></span>
+            </div>
+            <p class="settings-desc" style="margin-top:10px;margin-bottom:0;">Leave blank to allow all scopes. Use presets to start with least privilege.</p>
+          </div>
+
+          <div class="settings-section">
+            <h3 class="settings-title">Configuration</h3>
             <p class="settings-desc">Manage your full config by editing <code>~/.zubo/config.json</code> directly, or re-run <code>zubo setup</code>.</p>
           </div>
         </div>
@@ -2841,20 +2888,30 @@ function renderMemoryItems(results, container) {
     return;
   }
   document.getElementById('memory-count').textContent = String(results.length);
-  results.forEach(function(r) {
-    var item = document.createElement('div');
-    item.className = 'memory-item';
-    var src = document.createElement('div');
-    src.className = 'source';
-    src.textContent = r.source || '';
-    var cnt = document.createElement('div');
-    cnt.className = 'content';
-    cnt.textContent = r.content;
-    item.appendChild(src);
-    item.appendChild(cnt);
-    container.appendChild(item);
-  });
-}
+  results.forEach(function(r) {
+    var item = document.createElement('div');
+    item.className = 'memory-item';
+    var src = document.createElement('div');
+    src.className = 'source';
+    var sourceBits = [r.source || ''];
+    if (r.matchType) sourceBits.push(String(r.matchType));
+    if (typeof r.confidence === 'number') sourceBits.push('conf ' + Math.round(r.confidence * 100) + '%');
+    src.textContent = sourceBits.filter(Boolean).join(' • ');
+    var cnt = document.createElement('div');
+    cnt.className = 'content';
+    cnt.textContent = r.content;
+    item.appendChild(src);
+    if (r.reasons && r.reasons.length) {
+      var why = document.createElement('div');
+      why.className = 'source';
+      why.style.marginTop = '6px';
+      why.textContent = 'Reason: ' + r.reasons.join(', ');
+      item.appendChild(why);
+    }
+    item.appendChild(cnt);
+    container.appendChild(item);
+  });
+}
 
 function loadRecentMemories() {
   api('/memory/recent').then(function(data) {
@@ -3501,7 +3558,7 @@ function wipeData(type) {
 // --- SETTINGS ---
 var settingsProviders = [];
 
-function loadSettings() {
+function loadSettings() {
   api('/config').then(function(data) {
     settingsProviders = data.providers || [];
     var sel = document.getElementById('settings-provider');
@@ -3520,11 +3577,13 @@ function loadSettings() {
     document.getElementById('settings-heartbeat').value = data.minutes || 30;
     document.getElementById('heartbeat-status').textContent = '';
   });
-  loadChannelStatus();
-  loadDbStats();
-  loadSecrets();
-  loadSmartRouting();
-}
+  loadChannelStatus();
+  loadDbStats();
+  loadSecrets();
+  loadSmartRouting();
+  loadMemoryRetrievalSettings();
+  loadToolScopeSettings();
+}
 
 function onProviderChange() {
   var sel = document.getElementById('settings-provider');
@@ -3572,7 +3631,7 @@ function testLlm() {
   });
 }
 
-function saveHeartbeat() {
+function saveHeartbeat() {
   var mins = parseInt(document.getElementById('settings-heartbeat').value, 10);
   if (!mins || mins < 1 || mins > 1440) {
     document.getElementById('heartbeat-status').textContent = 'Must be 1\u20131440 minutes';
@@ -3590,7 +3649,90 @@ function saveHeartbeat() {
       document.getElementById('heartbeat-status').textContent = data.error || 'Error';
     }
   });
-}
+}
+
+function loadMemoryRetrievalSettings() {
+  api('/settings/memory-retrieval').then(function(data) {
+    document.getElementById('memory-context-topk').value = data.contextTopK || 3;
+    document.getElementById('memory-min-confidence').value = data.minConfidence || 0;
+    document.getElementById('memory-retrieval-status').textContent = '';
+  });
+}
+
+function saveMemoryRetrievalSettings() {
+  var contextTopK = parseInt(document.getElementById('memory-context-topk').value, 10);
+  var minConfidence = parseFloat(document.getElementById('memory-min-confidence').value);
+  if (isNaN(contextTopK) || contextTopK < 1 || contextTopK > 10) {
+    document.getElementById('memory-retrieval-status').textContent = 'Top-K must be 1-10';
+    return;
+  }
+  if (isNaN(minConfidence) || minConfidence < 0 || minConfidence > 1) {
+    document.getElementById('memory-retrieval-status').textContent = 'Confidence must be 0-1';
+    return;
+  }
+  api('/settings/memory-retrieval', {
+    method: 'PUT',
+    headers: {'Content-Type':'application/json'},
+    body: JSON.stringify({ contextTopK: contextTopK, minConfidence: minConfidence })
+  }).then(function(data) {
+    if (data.ok) {
+      document.getElementById('memory-retrieval-status').textContent = 'Saved';
+      toast('Memory retrieval settings updated');
+    } else {
+      document.getElementById('memory-retrieval-status').textContent = data.error || 'Error';
+    }
+  });
+}
+
+function applyMemoryPreset(kind) {
+  if (kind === 'strict') {
+    document.getElementById('memory-context-topk').value = 2;
+    document.getElementById('memory-min-confidence').value = 0.35;
+  } else {
+    document.getElementById('memory-context-topk').value = 4;
+    document.getElementById('memory-min-confidence').value = 0.2;
+  }
+  document.getElementById('memory-retrieval-status').textContent = 'Preset applied';
+}
+
+function loadToolScopeSettings() {
+  api('/settings/tool-scopes').then(function(data) {
+    document.getElementById('tool-scopes-allowed').value = (data.allowed || []).join(',');
+    document.getElementById('tool-scopes-dry-run').value = data.dryRunByDefault ? 'true' : 'false';
+    document.getElementById('tool-scopes-status').textContent = '';
+  });
+}
+
+function saveToolScopeSettings() {
+  var allowed = document.getElementById('tool-scopes-allowed').value
+    .split(',')
+    .map(function(x) { return x.trim(); })
+    .filter(Boolean);
+  var dryRunByDefault = document.getElementById('tool-scopes-dry-run').value === 'true';
+  api('/settings/tool-scopes', {
+    method: 'PUT',
+    headers: {'Content-Type':'application/json'},
+    body: JSON.stringify({ allowed: allowed, dryRunByDefault: dryRunByDefault })
+  }).then(function(data) {
+    if (data.ok) {
+      document.getElementById('tool-scopes-status').textContent = 'Saved';
+      toast('Tool safety settings updated');
+    } else {
+      document.getElementById('tool-scopes-status').textContent = data.error || 'Error';
+    }
+  });
+}
+
+function applyToolScopePreset(kind) {
+  if (kind === 'safe') {
+    document.getElementById('tool-scopes-allowed').value = 'memory,network_read,filesystem_read,config,scheduling';
+    document.getElementById('tool-scopes-dry-run').value = 'true';
+  } else {
+    document.getElementById('tool-scopes-allowed').value = 'memory,network_read,filesystem_read,filesystem_write,config,scheduling,delegation';
+    document.getElementById('tool-scopes-dry-run').value = 'true';
+  }
+  document.getElementById('tool-scopes-status').textContent = 'Preset applied';
+}
 
 // --- Smart Routing ---
 function loadSmartRouting() {