@pheem49/mint 1.5.0 → 1.5.2

package/README.md

@@ -21,12 +21,16 @@ Mint is an AI assistant built to live in your desktop and terminal. It combines
 ## What's New
 
 - **Unified CLI Agent:** `mint` now routes every normal message through the same agent loop. It can think, answer conversationally, inspect projects, edit files, run tools, and finish directly for simple chat.
+- **Fast Mode:** `/fast` switches the interactive CLI into a quieter `[Fast]` status that keeps the working indicator visible but hides internal `Thinking:` and tool-progress trace messages.
+- **Live CLI Replies:** Mint responses now appear in one live-updating `Mint` message instead of waiting for the whole final answer to render at once.
+- **Learned Skills:** `mint learn <path>` and `/learn <path>` import local `.md` or `.txt` files as persistent skill/instruction memory. Learned skills can be listed and deleted.
 - **Provider Fallback:** The agent can fall back across supported providers, for example from local OpenAI-compatible backends to Gemini.
 - **Provider Visibility:** Desktop and CLI responses show the provider/model that actually answered, including fallback results.
+- **Live2D Assistant Model:** Desktop UI now supports a Live2D Shiroko model with expression cycling, lip sync while speaking, transparent interaction-area overlays, and click-to-chat reactions.
 - **Google Workspace + Notion Integrations:** Gmail, Google Calendar, and Notion plugins can be configured from onboarding.
 - **Safety Manager:** Central safety policy for shell commands and actions, including deterministic command blocking, permission tiers, path guards, and action logs.
 - **Refactored Main Process:** Electron startup is split into focused modules for windows, IPC, proactive loop, screen capture, and action execution.
-- **CI & Audit Baseline:** GitHub Actions runs install, tests, and security audit. Current local test baseline is `120` passing tests and `0` high vulnerabilities.
+- **CI & Audit Baseline:** GitHub Actions runs install, tests, and security audit. Current local test baseline is `137` passing tests and `0` high vulnerabilities.
 - **Dependency Hardening:** Removed vulnerable `google-tts-api` and `xlsx`; replaced with internal Google TTS URL generation and `read-excel-file`.
 
 ## Key Features
@@ -36,6 +40,8 @@ Mint is an AI assistant built to live in your desktop and terminal. It combines
 Mint CLI is not just a chat wrapper. It is a workspace-aware agent loop.
 
 - **Think Before Acting:** Every request goes through an agent decision step.
+- **Fast Mode:** Toggle `/fast` to hide internal thought/progress messages while keeping the final answer, approvals, tools, and working indicator unchanged.
+- **Live Answer Rendering:** Final answers are streamed into a single Mint message block as they arrive.
 - **Conversational + Coding in One Flow:** Casual messages can finish directly; coding tasks can inspect, plan, edit, and verify.
 - **Workspace Context:** Reads current path, git status, diff summary, package scripts, and previous workspace session memory.
 - **Tool Use:** Supports web search, file listing, file reading, code search, path finding, shell commands, patch edits, file writes, opening folders, and asking the user.
@@ -46,6 +52,11 @@ Mint CLI is not just a chat wrapper. It is a workspace-aware agent loop.
 ### Desktop Assistant
 
 - **Electron Desktop UI:** Transparent desktop assistant window with tray support.
+- **Live2D Model View:** Optional Live2D assistant panel with model show/hide persistence. New installs start with the model hidden until the user enables it.
+- **Live2D Expressions:** Cycle model expressions from the toolbar and show an on-canvas expression toast for the active expression.
+- **Click Reactions:** Named model interaction zones (`Head Pat`, `Cheek Poke`, `Hand Tap`, `Shoulder Tap`, and `Careful`) can trigger temporary expressions and send short contextual prompts into the normal chat flow.
+- **Interaction Guide Overlay:** Toggle a transparent overlay that labels clickable model areas without blocking pointer input.
+- **Voice Lip Sync:** When Mint speaks, Live2D mouth parameters animate during TTS playback and reset when speech ends.
 - **Floating Widget:** Always-on-top quick access widget.
 - **Spotlight Launcher:** `Alt+Space` quick prompt window.
 - **Screen Vision:** Capture the screen and send selected regions to the AI.
@@ -69,6 +80,7 @@ Mint CLI is not just a chat wrapper. It is a workspace-aware agent loop.
 - **Chat History:** Persistent local chat transcript.
 - **Behavior Memory:** Stores recurring user context for proactive suggestions.
 - **Long-Term Memory Store:** SQLite-backed user context, session memories, usage patterns, and response cache.
+- **Learned Skill Files:** Import `.md` or `.txt` instruction files with `mint learn <path>` or `/learn <path>`. Mint remembers them as persistent skill/instruction context.
 - **Knowledge Base / RAG:** Index and search local `.txt`, `.md`, `.pdf`, `.docx`, and `.xlsx` files.
 - **Workspace Session Memory:** Remembers previous task summary and verification for each workspace.
 
@@ -167,7 +179,18 @@ npm start
 
 - `mint` / `mint chat` - Start the unified interactive agent UI.
 - `mint chat "<message>"` - Start with an initial message.
+- `mint chat --image ./screenshot.png "What is on this screen?"` - Attach an image to the initial chat message.
+- `/image ./screenshot.png What is on this screen?` - Attach an image while inside the interactive CLI, then press Enter to send.
+- `Ctrl+V` or `/paste What is on this screen?` - Attach clipboard images inside the interactive CLI, then press Enter to send.
+- `mint learn ./skill.md` - Read a local `.md` or `.txt` file and remember it as a persistent Mint skill/instruction.
+- `mint learn --list` - List learned skill files.
+- `mint learn --delete <id|path|name>` - Delete a learned skill by ID, path, or file name.
+- `mint summarize [path]` - Summarize repository structure, package metadata, git state, and key files.
+- `mint symbols [path]` - Build a local source symbol index for supported code files.
+- `mint semantic-code index [path]` - Create embeddings for local source code chunks.
+- `mint semantic-code search "<query>"` - Search the indexed code semantically.
 - `mint code "<task>"` - Run a specific coding task in the current workspace.
+- `mint code --image ./mockup.png "Build this UI"` - Attach an image as visual context for a coding task.
 - `mint gmail auth` - Open Google OAuth and save a Gmail refresh token.
 - `mint gmail auth --no-open` - Print the Gmail OAuth link without opening a browser.
 - `mint task "<task>"` - Queue a background task for the headless agent.
@@ -246,9 +269,20 @@ The plugin can create pages, query database pages, and append text blocks.
 Inside `mint`:
 
 - `/help` - Show commands.
+- `/fast [on|off|status]` - Toggle Fast Mode. Fast Mode shows `[Fast]`, keeps `Mint is thinking...`, and hides `Thinking:`/progress trace messages.
+- `/summarize [path] [--json]` - Summarize the current repository without using the AI agent.
+- `/symbols [path] [--json] [--limit n]` - Build a local source symbol index without using the AI agent.
+- `/semantic-code index` - Create embeddings for source code chunks in the current workspace.
+- `/semantic-code search <query>` - Search the indexed code semantically.
+- `/learn <path>` - Read a local `.md` or `.txt` file and remember it as a persistent Mint skill/instruction.
+- `/memory skills` - Show learned skill files.
+- `/memory skills delete <id|path|name>` - Delete a learned skill.
+- `/image <path> [prompt]` - Attach an image from disk.
+- `/paste [prompt]` - Attach an image from the clipboard.
 - `/code <task>` - Force Code Mode.
 - `/cd <path>` - Change active workspace directory.
 - `/models [name]` - Show or switch model/provider.
+- `/memory [cmd]` - Manage long-term memory.
 - `/config` - Show current configuration.
 - `/copy` - Copy last response.
 - `/clear` / `/reset` - Clear conversation history.

package/main.js

@@ -1,11 +1,9 @@
 const { app, BrowserWindow, ipcMain, shell, globalShortcut, clipboard } = require('electron');
 require('dotenv').config();
 
-const { handleChat, resetChat, getChatTranscript, translateImageContent, refreshApiKeyFromConfig } = require('./src/AI_Brain/Gemini_API');
 const { getSystemInfo, getWeather } = require('./src/System/system_info');
 const { readConfig, writeConfig } = require('./src/System/config_manager');
 const { parseCommand } = require('./src/Command_Parser/parser');
-const { executeAction } = require('./src/System/action_executor');
 const { getGoogleTtsUrls } = require('./src/System/google_tts_urls');
 const { createWindowManager } = require('./src/System/window_manager');
 const { createProactiveLoop } = require('./src/System/proactive_loop');
@@ -16,6 +14,18 @@ const systemEvents = require('./src/System/system_events');
 const customWorkflows = require('./src/System/custom_workflows');
 const mcpManager = require('./src/Plugins/mcp_manager');
 
+let geminiServices = null;
+function getGeminiServices() {
+    if (!geminiServices) {
+        geminiServices = require('./src/AI_Brain/Gemini_API');
+    }
+    return geminiServices;
+}
+
+function getActionExecutor() {
+    return require('./src/System/action_executor');
+}
+
 const projectRoot = __dirname;
 const windowManager = createWindowManager(projectRoot);
 const proactiveLoop = createProactiveLoop({
@@ -26,7 +36,7 @@ const proactiveLoop = createProactiveLoop({
 });
 const screenCapture = createScreenCaptureController({
     projectRoot,
-    translateImageContent,
+    translateImageContent: (...args) => getGeminiServices().translateImageContent(...args),
     getMainWindow: windowManager.getMainWindow
 });
 
@@ -39,16 +49,16 @@ registerIpcHandlers({
     proactiveLoop,
     screenCapture,
     services: {
-        handleChat,
-        resetChat,
-        getChatTranscript,
-        refreshApiKeyFromConfig,
+        handleChat: (...args) => getGeminiServices().handleChat(...args),
+        resetChat: (...args) => getGeminiServices().resetChat(...args),
+        getChatTranscript: (...args) => getGeminiServices().getChatTranscript(...args),
+        refreshApiKeyFromConfig: (...args) => getGeminiServices().refreshApiKeyFromConfig(...args),
         getSystemInfo,
         getWeather,
         readConfig,
         writeConfig,
         parseCommand,
-        executeAction,
+        executeAction: (...args) => getActionExecutor().executeAction(...args),
         getGoogleTtsUrls,
         customWorkflows
     }
@@ -59,14 +69,18 @@ app.whenReady().then(() => {
     const mainWindow = windowManager.createMainWindow();
     windowManager.createTray();
 
-    if (config.showDesktopWidget !== false) {
-        windowManager.createWidgetWindow();
-    }
+    mainWindow.once('ready-to-show', () => {
+        if (config.showDesktopWidget !== false) {
+            setTimeout(() => windowManager.createWidgetWindow(), 300);
+        }
 
-    mcpManager.init().catch(err => console.error('[MCP] Init Error:', err));
+        setTimeout(() => {
+            mcpManager.init().catch(err => console.error('[MCP] Init Error:', err));
 
-    const bridgeManager = require('./src/System/bridge_manager');
-    bridgeManager.init().catch(err => console.error('[BridgeManager] Init Error:', err));
+            const bridgeManager = require('./src/System/bridge_manager');
+            bridgeManager.init().catch(err => console.error('[BridgeManager] Init Error:', err));
+        }, 1000);
+    });
 
     systemEvents.startMonitoring();
     if (config.enableCustomWorkflows !== false) {

package/mint-cli-logic.js

@@ -1,119 +1,3 @@
-// Mint CLI Action Logic
-const { openApp } = require('./src/Automation_Layer/open_app');
-const { openWebsite, openSearch } = require('./src/Automation_Layer/open_website');
-const { createFolder, openFile, deleteFile, findPath } = require('./src/Automation_Layer/file_operations');
-const safetyManager = require('./src/System/safety_manager');
-
-async function executeAction(action, options = {}) {
-    if (!action || action.type === 'none') return null;
-
-    try {
-        const safety = safetyManager.assertActionAllowed(action, {
-            allowDangerous: options.allowDangerous === true
-        });
-        safetyManager.appendActionLog({
-            source: options.source || 'mint_cli_logic',
-            action: action.type,
-            target: action.target || action.path || '',
-            tier: safety.tier,
-            approved: options.allowDangerous === true || safety.tier !== safetyManager.TIERS.DANGEROUS
-        });
-
-        switch (action.type) {
-            case 'open_url':
-                openWebsite(action.target);
-                return `Opened URL: ${action.target}`;
-            case 'search':
-                openSearch(action.target);
-                return `Searching for: ${action.target}`;
-            case 'open_app':
-                openApp(action.target);
-                return `Opening app: ${action.target}`;
-            case 'create_folder':
-                createFolder(action.target);
-                return `Created folder: ${action.target}`;
-            case 'open_file':
-            case 'open_folder':
-                const res = await openFile(action.target);
-                return res === true ? `Opening: ${action.target}` : res;
-            case 'delete_file':
-                await deleteFile(action.target);
-                return `Deleted: ${action.target}`;
-            case 'find_path': {
-                const result = findPath(action.target, {
-                    type: action.pathType,
-                    maxResults: 10
-                });
-                if (!result.success) {
-                    return result.message;
-                }
-
-                if (action.openAfter === true) {
-                    if (result.matches.length === 1) {
-                        const match = result.matches[0];
-                        await openFile(match.path);
-                        return `Found and opened ${match.type === 'dir' ? 'folder' : 'file'}: ${match.path}`;
-                    }
-                    return `Found multiple matches for "${action.target}". Please be more specific:\n${result.matches.map(m => `- [${m.type}] ${m.path}`).join('\n')}`;
-                }
-
-                return `Found matches for "${action.target}":\n${result.matches.map(m => `- [${m.type}] ${m.path}`).join('\n')}`;
-            }
-            case 'learn_file':
-                return await indexFile(action.target);
-            case 'mcp_tool':
-                const mcpManager = require('./src/Plugins/mcp_manager');
-                const mcpResult = await mcpManager.callTool(action.server, action.target, action.args);
-                return JSON.stringify(mcpResult.content);
-            case 'mouse_move':
-                const granularAutomation = require('./src/System/granular_automation');
-                return await granularAutomation.mouseMove(action.x, action.y);
-            case 'mouse_click':
-                const granularAutomationClick = require('./src/System/granular_automation');
-                return await granularAutomationClick.mouseClick(action.x, action.y, action.button || 1);
-            case 'type_text':
-                const granularAutomationType = require('./src/System/granular_automation');
-                return await granularAutomationType.typeText(action.target);
-            case 'key_tap':
-                const granularAutomationKey = require('./src/System/granular_automation');
-                return await granularAutomationKey.keyTap(action.target);
-            case 'plugin':
-                return await pluginManager.executePlugin(action.pluginName, action.target);
-            case 'system_automation':
-                return await handleSystemAutomation(action.target);
-            case 'system_info':
-                return await SystemAutomation.getSystemInfo(action.target);
-            default:
-                return `Action ${action.type} is not yet fully supported in CLI.`;
-        }
-    } catch (err) {
-        return `Error executing action: ${err.message}`;
-    }
-}
-
-async function handleSystemAutomation(target) {
-    const [cmd, value] = target.split(':');
-    switch (cmd) {
-        case 'volume':
-            return await SystemAutomation.setVolume(parseInt(value));
-        case 'mute':
-            return await SystemAutomation.mute();
-        case 'brightness':
-            return await SystemAutomation.setBrightness(parseInt(value));
-        case 'sleep':
-            return await SystemAutomation.sleep();
-        case 'restart':
-            return await SystemAutomation.restart();
-        case 'shutdown':
-            return await SystemAutomation.shutdown();
-        case 'minimize_all':
-            return await SystemAutomation.minimizeAll();
-        default:
-            if (SystemAutomation[target]) {
-                return await SystemAutomation[target]();
-            }
-            throw new Error(`Unknown system command: ${target}`);
-    }
-}
-
-module.exports = { executeAction };
+// Backward-compatible CLI entry point for Mint actions.
+// Keep execution rules in one place so desktop, CLI, and agents share safety behavior.
+module.exports = require('./src/System/action_executor');