agent-world 0.12.3 → 0.15.0

package/README.md

@@ -37,25 +37,31 @@ Paste that prompt. Agents come alive instantly.
 - ✅ Built-in Rules for Messages - Turn limits to prevent loops
 - ✅ Concurrent Chat Sessions - Isolated `chatId` routing enables parallel conversations
 - ✅ Progressive Agent Skills - Skills are discovered and loaded on demand via `load_skill`
+- ✅ World-Level Tool Permissions - `Read`, `Ask`, and `Auto` modes control write and execution capabilities
 - ✅ Cross-Client HITL Approval - Option-based approvals in CLI, Web, and Electron
+- ✅ Reasoning Controls - World-scoped reasoning effort with separate reasoning-token rendering
+- ✅ Heartbeats & Queueing - Queue-backed world prompts and cron scheduling with explicit controls
+- ✅ Marketplace & Imports - Bring in worlds, agents, and skills from local folders or curated GitHub sources
 - ✅ Runtime Controls - Session-scoped send/stop flows and tool lifecycle visibility
 - ✅ Safer Tool Execution - Trusted-CWD and argument-scope guards for `shell_cmd`
 - ✅ Multiple AI Providers - Use different models for different agents
 - ✅ Web + CLI + Electron - Modern interfaces with real-time streaming and status feedback
 
-## Latest Highlights (since v0.11.1)
+## Latest Highlights (v0.15.0)
 
-- Web app now includes a Settings page, chat-history search, and branch-from-message workflow
-- Built-in `create_agent` tool now supports approval-first creation and inherits world chat defaults
-- File exploration is easier with built-in `read_file`, `list_files` (including recursive mode), and `grep`
-- Electron now supports folder-based world import/export with validation and conflict handling
-- Chat UX is smoother with clearer inline working status, better streaming feedback, and UI refinements
-- Real-time refresh behavior is improved across web and desktop when agents/world data changes
+- World-level tool permissions add `Read`, `Ask`, and `Auto` control modes in both the web and Electron composers
+- Supported providers can use world-scoped `reasoning_effort`, with reasoning tokens streamed and rendered separately from final answers
+- World and heartbeat prompts now enter the same queue-backed message flow as human turns, with explicit heartbeat controls and next-run visibility
+- Electron now includes marketplace-style import flows for discovering and importing worlds, agents, and skills
+- Web and Electron transcripts have tighter parity with compact tool rows, improved status rendering, and cleaner working indicators
+- Clicking markdown links in Electron messages now opens the external browser instead of attempting in-app navigation
 
 ## Release Notes
 
+- **v0.15.0** - Tool permission modes, reasoning-token support, queue-backed world/heartbeat flow, marketplace imports, and transcript/UI refinements
+- **v0.14.0** - User message queue, `write_file`/`web_fetch`/`send_message`, shell risk gating, E2E harnesses, skill editor, and heartbeat scheduling
+- **v0.13.0** - Better HITL prompts, improved streaming/main-agent UX, and smoother message rendering
 - **v0.12.0** - Web settings/search/branching, built-in `create_agent`, new file tools, Electron folder import/export, and chat/status UX improvements
-- **v0.11.0** - Electron desktop workflow, concurrent chat sessions, main-agent routing, progressive skills + HITL, and runtime safety hardening
 - Full history: [CHANGELOG.md](CHANGELOG.md)
 
 ## What You Can Build
@@ -102,25 +108,24 @@ Each Agent World has a collection of agents that can communicate through a share
 
 ### Message Rules
 
-| Message Type | Example | Who Responds |
+| Message Shape | Example | Who Responds |
 |--------------|---------|--------------|
-| **Human message** | `Hello everyone!` | All active agents |
-| **Direct mention** | `@alice Can you help?` | Only @alice |
-| **Paragraph mention** | `Please review this:\n@alice` | Only @alice |
-| **Mid-text mention** | `I think @alice should help` | Nobody (event is persisted; no agent-memory save) |
+| **Public human or world message** | `Hello everyone!` | All active agents |
+| **Paragraph-start mention** | `@alice Can you help?` | Only mentioned agents |
+| **Paragraph-start mention after text** | `Please review this:\n@alice` | Only mentioned agents |
+| **Mid-text mention only** | `I think @alice should help` | Nobody (event is persisted; no agent-memory save) |
 | **Stop World** | `<world>pass</world>` | No agents |
 
 ### Agent Behavior
 
 **Agents always respond to:**
-- Human messages (unless mentioned agents exist)
+- Public human or world messages when there is no paragraph-start mention
 - Direct @mentions at paragraph start
-- World messages
 
 **Agents never respond to:**
 - Their own messages
 - Other agents (unless @mentioned at paragraph start)
-- Mid-text mentions (not at paragraph start)
+- Mid-text mentions (not at paragraph start), including world prompts
 
 **When messages are saved to agent memory:**
 - Incoming messages are saved only for agents that will respond
@@ -169,7 +174,13 @@ npm run electron:dev
 
 ## Project Structure
 
-See [Project Structure Documentation](project.md)
+- `core/` - shared runtime, storage, tools, provider integrations, and event flow
+- `server/` - REST API and SSE transport
+- `web/` - browser app
+- `electron/` - desktop app (main, preload, renderer)
+- `cli/` - terminal interface
+
+For broader architecture and usage docs, start with [Docs Home](docs/docs-home.md).
 
 ## Development Scripts
 
@@ -237,6 +248,38 @@ AZURE_OPENAI_API_VERSION=2024-10-21-preview
 OLLAMA_BASE_URL=http://localhost:11434
 ```
 
+### Optional Opik Layer
+
+Opik is optional and fully gated.
+
+```bash
+OPIK_ENABLED=false
+OPIK_SAFETY_ENABLED=false
+OPIK_EVAL_ENABLED=false
+OPIK_API_KEY=
+OPIK_WORKSPACE=
+OPIK_PROJECT=agent-world-debugging
+```
+
+Rules:
+- `OPIK_ENABLED=false`: all Opik integration/safety/eval paths are inert.
+- `OPIK_ENABLED=true`: tracing can attach only if `OPIK_API_KEY` + `OPIK_WORKSPACE` are set.
+- Safety and eval still require their sub-flags (`OPIK_SAFETY_ENABLED`, `OPIK_EVAL_ENABLED`).
+
+#### Fallback Behavior
+
+When Opik is enabled but something is missing, startup always continues normally — no crashes.
+
+| Condition | Result | Log |
+|-----------|--------|-----|
+| `OPIK_ENABLED=false` | No Opik code runs | none |
+| Enabled but `OPIK_API_KEY` or `OPIK_WORKSPACE` missing | Tracer skipped | warning: `Opik enabled but required env is missing` |
+| Enabled + config present but `packages/opik` not installed | Tracer skipped | warning: `Opik enabled but optional dependency is unavailable` |
+| Enabled + config present + module loaded but tracer init fails | Tracer skipped | warning: `Opik module loaded but tracer initialization failed` |
+| Enabled + config present + module loaded + tracer created | Tracer attaches to world | info: `Opik tracer attached` |
+
+Opik is storage-agnostic — it attaches to `world.eventEmitter` and works identically with sqlite, file, or memory storage backends.
+
 ## Testing
 
 **Run all tests:**
@@ -245,6 +288,7 @@ npm test              # Run all unit tests
 npm run test:watch    # Watch mode with hot reload
 npm run test:ui       # Visual test UI
 npm run test:coverage # Generate coverage report
+npm run test:coverage:gate # Coverage + core threshold gate + subsystem scorecard
 ```
 
 **Run specific tests:**
@@ -256,10 +300,23 @@ npm test -- message-saving      # Test files matching pattern
 **Integration tests:**
 ```bash
 npm run test:integration  # Run integration tests with real filesystem
+npm run ci:test           # CI gate: coverage threshold + integration
 ```
 
 Agent World uses Vitest for fast, modern testing with native TypeScript support.
 
+### Coverage Gates
+
+`npm run test:coverage:gate` enforces minimum core coverage thresholds and generates a subsystem scorecard:
+- core statements >= 68%
+- core branches >= 56%
+- core functions >= 75%
+- core lines >= 69%
+
+The scorecard is written to:
+- `coverage/scorecard.md`
+- `coverage/scorecard.json`
+
 ## Logging and Debugging
 
 Agent World uses **scenario-based logging** to help you debug specific issues without noise. Enable only the logs you need for your current task.
@@ -275,6 +332,9 @@ LOG_MCP=debug npm run web:dev
 
 # Agent response debugging
 LOG_EVENTS_AGENT=debug LOG_LLM=debug npm run web:dev
+
+# Chat restore/HITL replay debugging
+LOG_CHAT_RESTORE=debug LOG_CHAT_RESTORE_RESUME=debug LOG_CHAT_RESTORE_RESUME_TOOLS=debug LOG_HITL=debug npm run web:dev
 ```
 
 **For complete logging documentation**, see [Logging Guide](docs/logging-guide.md).
@@ -299,6 +359,7 @@ export AGENT_WORLD_DATA_PATH=./data/worlds
 - **[Building Agents with Just Words](docs/Building%20Agents%20with%20Just%20Words.md)** - Complete guide with examples
 - **[Shell Command Tool (shell_cmd)](docs/shell-cmd-tool.md)** - Built-in tool for executing shell commands
 - **[HITL Approval Flow](docs/hitl-approval-flow.md)** - Option-based approval flow across Core/Electron/Web/CLI
+- **[API Reference](openapi.yaml)** - OpenAPI 3.1 spec for the REST API
 - **[Using Core from npm](docs/core-npm-usage.md)** - Integration guide for server and browser apps
 - **[Electron Desktop App](docs/electron-desktop.md)** - Open-folder workflow and local world creation
 
@@ -307,6 +368,17 @@ export AGENT_WORLD_DATA_PATH=./data/worlds
 
 Agent World includes built-in tools that are automatically available to all agents:
 
+- `read_file`, `list_files`, `grep` - inspect project files and source trees
+- `write_file` - update files inside trusted workspace scope
+- `web_fetch` - fetch web pages and convert them to Markdown for agent use
+- `send_message` - dispatch trusted chat-context messages to other agents
+- `create_agent` - create new agents during a conversation
+- `shell_cmd` - run shell commands with trusted-scope validation and lifecycle controls
+- `load_skill` - progressively load `SKILL.md` instructions and related scripts
+- `human_intervention_request` - ask the user for structured options and approvals
+
+World owners can set built-in tool access to `Read`, `Ask`, or `Auto` so write/execute actions are blocked, approval-gated, or automatic depending on the world.
+
 ### shell_cmd
 Execute shell commands with full output capture and execution history. Perfect for file operations, system information, and automation tasks.
 
@@ -333,6 +405,7 @@ Agent World includes progressive skill loading through the `load_skill` built-in
   - User roots: `~/.agents/skills`, `~/.codex/skills`
 - The model receives compact skill summaries first, then calls `load_skill` only when full instructions are needed.
 - Skill activation in interactive runtimes is HITL-gated.
+- `load_skill` always performs the same preflight flow: script references are discovered from instructions, script execution is HITL-approved and scope-validated, and reference-file context is collected when active resources are present.
 
 Minimal `SKILL.md` example:
 
@@ -355,6 +428,21 @@ HITL options for skill activation:
 - `yes_in_session`: approve this `skill_id` in the current world/chat session
 - `no`: decline
 
+### human_intervention_request (Generic Human Input)
+
+`human_intervention_request` lets the model ask a human question, present options, and optionally require explicit confirmation.
+The tool is options-only (no free-text mode).
+
+Option example:
+
+```json
+{
+  "question": "Choose deployment strategy",
+  "options": ["Blue/Green", "Canary", "Rolling"],
+  "requireConfirmation": true
+}
+```
+
 ## Experimental Features
 
 - **MCP Support** - *Currently in experiment* - Model Context Protocol integration for tools like search and code execution. e.g.,

package/dist/cli/commands.d.ts

@@ -43,6 +43,11 @@
  * - Creates target directory if it doesn't exist
  * - Supports migration between storage types
  * - Event history preserved across different storage backends
+ *
+ * Recent Changes:
+ * - 2026-03-06: Removed runtime message/export fallback to `world.currentChatId`; CLI execution now requires explicit selected chat state.
+ * - 2026-03-10: Switched CLI user message send path to the queue-only `enqueueAndProcessUserTurn` API.
+ * - 2026-03-04: Added queue metadata (`messageId`, `queueStatus`, `queueRetryCount`) to successful message-send CLI results for queue error visibility.
  */
 import { World } from '../core/types.js';
 export interface CLIResponse {
@@ -57,6 +62,7 @@ export interface CLIResponse {
 export interface CLIContext {
     currentWorldName?: string;
     currentWorld?: World | null;
+    selectedChatId?: string | null;
 }
 export type PromptFunction = (question: string, options?: string[]) => Promise<string>;
 /**
@@ -105,5 +111,5 @@ export declare function deleteExistingData(targetPath: string, storageType: 'fil
 }>;
 export declare function performWorldSave(world: World, storageType: 'file' | 'sqlite', targetPath: string): Promise<CLIResponse>;
 export declare function processCLICommand(input: string, context: CLIContext, promptFn: PromptFunction): Promise<CLIResponse>;
-export declare function processCLIInput(input: string, world: World | null, sender?: string): Promise<CLIResponse>;
+export declare function processCLIInput(input: string, world: World | null, sender?: string, selectedChatId?: string | null): Promise<CLIResponse>;
 export {};

package/dist/cli/commands.js

@@ -43,8 +43,13 @@
  * - Creates target directory if it doesn't exist
  * - Supports migration between storage types
  * - Event history preserved across different storage backends
+ *
+ * Recent Changes:
+ * - 2026-03-06: Removed runtime message/export fallback to `world.currentChatId`; CLI execution now requires explicit selected chat state.
+ * - 2026-03-10: Switched CLI user message send path to the queue-only `enqueueAndProcessUserTurn` API.
+ * - 2026-03-04: Added queue metadata (`messageId`, `queueStatus`, `queueRetryCount`) to successful message-send CLI results for queue error visibility.
  */
-import { LLMProvider, createWorld, getWorld, updateWorld, publishMessage, listWorlds, deleteWorld, listAgents, getAgent, updateAgent, deleteAgent, createAgent, clearAgentMemory, listChats, updateChat, exportWorldToMarkdown, exportChatToMarkdown, newChat, restoreChat, deleteChat, getMemory } from '../core/index.js';
+import { LLMProvider, createWorld, getWorld, updateWorld, enqueueAndProcessUserTurn, listWorlds, deleteWorld, listAgents, getAgent, updateAgent, deleteAgent, createAgent, clearAgentMemory, listChats, updateChat, exportWorldToMarkdown, exportChatToMarkdown, newChat, restoreChat, deleteChat, getMemory } from '../core/index.js';
 import { createStorage } from '../core/storage/storage-factory.js';
 import { createCategoryLogger } from '../core/logger.js';
 import readline from 'readline';
@@ -83,7 +88,12 @@ const boldRed = (text) => `\x1b[1m\x1b[31m${text}\x1b[0m`;
  */
 export async function displayChatMessages(worldId, chatId) {
     try {
-        const messages = await getMemory(worldId, chatId);
+        const resolvedChatId = String(chatId || '').trim();
+        if (!resolvedChatId) {
+            console.log(gray('\n  No active chat selected.\n'));
+            return;
+        }
+        const messages = await getMemory(worldId, resolvedChatId);
         if (!messages || messages.length === 0) {
             console.log(gray('\n  No messages in current chat.\n'));
             return;
@@ -1900,7 +1910,7 @@ export async function processCLICommand(input, context, promptFn) {
                 const worldError = requireWorldOrError(world, command);
                 if (worldError)
                     return worldError;
-                const chatId = collectedParams.chatId || world.currentChatId;
+                const chatId = collectedParams.chatId || context.selectedChatId;
                 if (!chatId) {
                     cliResponse = {
                         success: false,
@@ -1946,10 +1956,11 @@ export async function processCLICommand(input, context, promptFn) {
     }
 }
 // Main CLI input processor - handles both commands and messages
-export async function processCLIInput(input, world, sender = 'human') {
+export async function processCLIInput(input, world, sender = 'human', selectedChatId = null) {
     const context = {
         currentWorld: world,
-        currentWorldName: world?.name
+        currentWorldName: world?.name,
+        selectedChatId,
     };
     // Simple prompt function for CLI
     const promptFunction = async (question, options) => {
@@ -1989,7 +2000,7 @@ export async function processCLIInput(input, world, sender = 'human') {
         };
     }
     try {
-        const currentChatId = String(world?.currentChatId || '').trim();
+        const currentChatId = String(selectedChatId || '').trim();
         if (!currentChatId) {
             return {
                 success: false,
@@ -1998,19 +2009,25 @@ export async function processCLIInput(input, world, sender = 'human') {
             };
         }
         const restoredWorld = await restoreChat(world.id, currentChatId);
-        if (!restoredWorld || restoredWorld.currentChatId !== currentChatId) {
+        if (!restoredWorld || !restoredWorld.chats?.has?.(currentChatId)) {
             return {
                 success: false,
                 message: `Cannot send message - chat not found: ${currentChatId}`,
                 technicalDetails: `Failed to restore active chat '${currentChatId}' before message publish`
             };
         }
-        publishMessage(world, input, sender, currentChatId);
+        const queuedMessage = await enqueueAndProcessUserTurn(world.id, currentChatId, input, sender, world);
         return {
             success: true,
             message: '',
-            data: { sender, chatId: currentChatId },
-            technicalDetails: `Message published to world '${world.name}' (chat '${currentChatId}')`
+            data: {
+                sender,
+                chatId: currentChatId,
+                messageId: queuedMessage?.messageId || null,
+                queueStatus: queuedMessage?.status || null,
+                queueRetryCount: typeof queuedMessage?.retryCount === 'number' ? queuedMessage.retryCount : null,
+            },
+            technicalDetails: `Message enqueued for world '${world.name}' (chat '${currentChatId}')`
         };
     }
     catch (error) {

package/dist/cli/hitl.d.ts

@@ -5,15 +5,17 @@
  * - Provide pure parsing/selection helpers for HITL option requests used by the CLI.
  *
  * Key Features:
- * - Parse `hitl-option-request` system payloads into normalized structures.
+ * - Parse HITL prompt payloads from tool-progress metadata and pending prompt envelopes.
  * - Resolve user input into option IDs (by number or option id).
  * - Provide deterministic fallback option resolution.
+ * - Guard duplicate replayed requests via requestId-tracking helper.
  *
  * Implementation Notes:
  * - Parser accepts generic event payloads and rejects incomplete requests.
- * - Fallback defaults to explicit `no` when available, otherwise first option.
+ * - Option fallback defaults to explicit `no` when available, otherwise first option.
  *
  * Recent Changes:
+ * - 2026-02-20: Enforced options-only HITL parsing in CLI helpers.
  * - 2026-02-14: Added initial helper module for CLI HITL response flow support.
  */
 export interface HitlOptionPayload {
@@ -26,8 +28,13 @@ export interface HitlOptionRequestPayload {
     title: string;
     message: string;
     chatId: string | null;
+    mode: 'option';
     options: HitlOptionPayload[];
     defaultOptionId: string;
 }
+export type HitlPromptRequestPayload = HitlOptionRequestPayload;
+export declare function parseHitlPromptRequest(eventData: unknown): HitlPromptRequestPayload | null;
+export declare function parseHitlPromptFromToolEvent(eventData: unknown): HitlPromptRequestPayload | null;
 export declare function parseHitlOptionRequest(eventData: unknown): HitlOptionRequestPayload | null;
 export declare function resolveHitlOptionSelectionInput(options: HitlOptionPayload[], rawInput: string, fallbackOptionId: string): string | null;
+export declare function markHitlRequestHandled(handledRequestIds: Set<string>, requestId: string): boolean;

package/dist/cli/hitl.js

@@ -5,34 +5,26 @@
  * - Provide pure parsing/selection helpers for HITL option requests used by the CLI.
  *
  * Key Features:
- * - Parse `hitl-option-request` system payloads into normalized structures.
+ * - Parse HITL prompt payloads from tool-progress metadata and pending prompt envelopes.
  * - Resolve user input into option IDs (by number or option id).
  * - Provide deterministic fallback option resolution.
+ * - Guard duplicate replayed requests via requestId-tracking helper.
  *
  * Implementation Notes:
  * - Parser accepts generic event payloads and rejects incomplete requests.
- * - Fallback defaults to explicit `no` when available, otherwise first option.
+ * - Option fallback defaults to explicit `no` when available, otherwise first option.
  *
  * Recent Changes:
+ * - 2026-02-20: Enforced options-only HITL parsing in CLI helpers.
  * - 2026-02-14: Added initial helper module for CLI HITL response flow support.
  */
-export function parseHitlOptionRequest(eventData) {
-    if (!eventData || typeof eventData !== 'object') {
-        return null;
-    }
-    const payload = eventData;
-    const content = payload.content && typeof payload.content === 'object'
-        ? payload.content
-        : null;
-    if (!content || String(content.eventType || '').trim() !== 'hitl-option-request') {
-        return null;
-    }
-    const requestId = String(content.requestId || '').trim();
+function normalizePromptPayload(promptData, fallbackChatId) {
+    const requestId = String(promptData.requestId || '').trim();
     if (!requestId) {
         return null;
     }
-    const options = Array.isArray(content.options)
-        ? content.options
+    const options = Array.isArray(promptData.options)
+        ? promptData.options
             .map((option) => {
             const optionRecord = option && typeof option === 'object'
                 ? option
@@ -48,19 +40,57 @@ export function parseHitlOptionRequest(eventData) {
     if (options.length === 0) {
         return null;
     }
-    const preferredDefault = String(content.defaultOptionId || '').trim();
+    const preferredDefault = String(promptData.defaultOptionId || '').trim();
     const defaultOptionId = options.some((option) => option.id === preferredDefault)
         ? preferredDefault
         : (options.find((option) => option.id === 'no')?.id || options[0].id);
     return {
         requestId,
-        title: String(content.title || 'Approval required').trim() || 'Approval required',
-        message: String(content.message || '').trim(),
-        chatId: payload.chatId ? String(payload.chatId) : null,
+        title: String(promptData.title || 'Approval required').trim() || 'Approval required',
+        message: String(promptData.message || '').trim(),
+        chatId: fallbackChatId,
+        mode: 'option',
         options,
         defaultOptionId
     };
 }
+export function parseHitlPromptRequest(eventData) {
+    if (!eventData || typeof eventData !== 'object') {
+        return null;
+    }
+    const payload = eventData;
+    const prompt = payload.prompt && typeof payload.prompt === 'object'
+        ? payload.prompt
+        : null;
+    if (!prompt) {
+        return null;
+    }
+    const chatId = payload.chatId ? String(payload.chatId) : null;
+    return normalizePromptPayload(prompt, chatId);
+}
+export function parseHitlPromptFromToolEvent(eventData) {
+    if (!eventData || typeof eventData !== 'object') {
+        return null;
+    }
+    const payload = eventData;
+    const toolExecution = payload.toolExecution && typeof payload.toolExecution === 'object'
+        ? payload.toolExecution
+        : null;
+    const metadata = toolExecution?.metadata && typeof toolExecution.metadata === 'object'
+        ? toolExecution.metadata
+        : null;
+    const prompt = metadata?.hitlPrompt && typeof metadata.hitlPrompt === 'object'
+        ? metadata.hitlPrompt
+        : null;
+    if (!toolExecution || !prompt) {
+        return null;
+    }
+    const chatId = prompt.chatId ? String(prompt.chatId) : (payload.chatId ? String(payload.chatId) : null);
+    return normalizePromptPayload(prompt, chatId);
+}
+export function parseHitlOptionRequest(eventData) {
+    return parseHitlPromptRequest(eventData);
+}
 export function resolveHitlOptionSelectionInput(options, rawInput, fallbackOptionId) {
     const normalizedInput = String(rawInput || '').trim();
     if (!normalizedInput) {
@@ -79,3 +109,14 @@ export function resolveHitlOptionSelectionInput(options, rawInput, fallbackOptio
     }
     return null;
 }
+export function markHitlRequestHandled(handledRequestIds, requestId) {
+    const normalizedRequestId = String(requestId || '').trim();
+    if (!normalizedRequestId) {
+        return false;
+    }
+    if (handledRequestIds.has(normalizedRequestId)) {
+        return false;
+    }
+    handledRequestIds.add(normalizedRequestId);
+    return true;
+}