open-agents-ai 0.187.259 → 0.187.262

package/README.md

@@ -35,32 +35,162 @@ An autonomous multi-turn tool-calling agent that reads your code, makes changes,
 - [How It Works](#how-it-works)
 - [Features](#features)
 - [Enterprise & Headless Mode](#enterprise--headless-mode)
+  - [Non-Interactive Mode](#non-interactive-mode)
+  - [Background Jobs](#background-jobs)
+  - [JSON Output Mode](#json-output-mode)
+  - [Process Management](#process-management)
+  - [REST API Service (Port 11435)](#rest-api-service-port-11435)
+    - [Access Policy & Binding](#access-policy--binding)
+    - [Working Directory](#working-directory)
+    - [Health & Observability](#health--observability)
+    - [OpenAI-Compatible Inference](#openai-compatible-inference)
+    - [Agentic Task Execution](#agentic-task-execution)
+    - [Configuration](#configuration-1)
+    - [Slash Commands via REST](#slash-commands-via-rest)
+    - [Auth Scopes](#auth-scopes)
+    - [Tool-Use Profiles](#tool-use-profiles)
+    - [Parallelism & Concurrency](#parallelism--concurrency)
+    - [Endpoint Reference](#endpoint-reference)
+    - [Stateful Chat — `/v1/chat` + `/api/chat` (OpenAI drop-in with full agent under the hood)](#stateful-chat--v1chat--apichat-openai-drop-in-with-full-agent-under-the-hood)
+    - [Live Comparison: Ollama vs OA Full Agent](#live-comparison-ollama-vs-oa-full-agent)
+    - [One-Off Completions — `/api/generate` + `/v1/generate`](#one-off-completions--apigenerate--v1generate)
+    - [Embeddings — `/v1/embeddings` + `/api/embed`](#embeddings--v1embeddings--apiembed)
+    - [Memory Recall + Knowledge Graph — `/v1/memory/*`](#memory-recall--knowledge-graph--v1memory)
+    - [Generate/Embed/Memory Test Harness](#generateembedmemory-test-harness)
+    - [AIWG Cascade — `/v1/aiwg/*`](#aiwg-cascade--v1aiwg)
+    - [ISO/IEC 42001:2023 AIMS — `/v1/aims/*`](#isoiec-420012023-aims--v1aims)
+    - [Event Bus — `/v1/events` (SSE fanout)](#event-bus--v1events-sse-fanout)
+    - [Memory + Skills + MCP + Tools + Engines (parity surface)](#memory--skills--mcp--tools--engines-parity-surface)
+    - [Sessions, Context, Cost, Sponsors, Nexus](#sessions-context-cost-sponsors-nexus)
+    - [RFC 7807 Problem Details (error envelope)](#rfc-7807-problem-details-error-envelope)
+    - [Pagination envelope](#pagination-envelope)
+    - [ETag + Conditional GET](#etag--conditional-get)
+    - [Web Interface](#web-interface)
 - [Architecture](#architecture)
 - [Context Engineering](#context-engineering)
 - [Model-Tier Awareness](#model-tier-awareness)
+  - [Small Model Optimization (Research-Backed)](#small-model-optimization-research-backed)
+  - [Tool Nesting for Small Models](#tool-nesting-for-small-models)
+  - [Dynamic Context Limits](#dynamic-context-limits)
 - [Live Code Knowledge Graph](#live-code-knowledge-graph)
+  - [How It Works](#how-it-works-1)
+  - [What the Agent Sees](#what-the-agent-sees)
+  - [Graph Tools](#graph-tools)
+  - [Storage](#storage)
+  - [Research Basis](#research-basis)
 - [Auto-Expanding Context Window](#auto-expanding-context-window)
 - [Tools (85+)](#tools-85)
 - [Model Context Protocol (MCP)](#model-context-protocol-mcp)
+  - [What MCP gives you](#what-mcp-gives-you)
+  - [Spec compliance — what we implement](#spec-compliance--what-we-implement)
+  - [Three ways to add a server](#three-ways-to-add-a-server)
+  - [Verified compatibility — 12 servers connect end-to-end](#verified-compatibility---12-servers-connect-end-to-end)
+  - [Streaming, progress, and binary content](#streaming-progress-and-binary-content)
+  - [Live agent eval](#live-agent-eval)
+  - [Programmatic API](#programmatic-api)
+  - [Further reading](#further-reading)
 - [Associative Memory & Cross-Modal Binding](#associative-memory--cross-modal-binding)
+  - [Architecture](#architecture-1)
+  - [Episode Store (SQLite)](#episode-store-sqlite)
+  - [Temporal Knowledge Graph](#temporal-knowledge-graph)
+  - [Zettelkasten Linking (A-MEM)](#zettelkasten-linking-a-mem)
+  - [PPR Retrieval (HippoRAG)](#ppr-retrieval-hipporag)
+  - [Cross-Modal Binding](#cross-modal-binding)
+  - [Gist Compression](#gist-compression)
+  - [Near-Critical Cognitive Architecture](#near-critical-cognitive-architecture)
+  - [Cross‑Modality Identity & Association (CLIP + Voice)](#crossmodality-identity--association-clip--voice)
 - [Ralph Loop — Iteration-First Design](#ralph-loop--iteration-first-design)
 - [Task Control](#task-control)
+  - [Pause, Stop, Resume, Destroy](#pause-stop-resume-destroy)
+  - [Session Context Persistence](#session-context-persistence)
+  - [Auto-Restore on Startup](#auto-restore-on-startup)
 - [COHERE Cognitive Framework](#cohere-cognitive-framework)
+  - [Distributed Inference (`/cohere`)](#distributed-inference-cohere)
+  - [How It Works](#how-it-works-2)
+  - [Research Provenance](#research-provenance)
 - [Context Compaction — Research-Backed Memory Management](#context-compaction--research-backed-memory-management)
+  - [How It Works](#how-it-works-3)
+  - [Compaction Strategies](#compaction-strategies)
+  - [Automatic Compaction](#automatic-compaction)
+  - [Deep Context Mode (`/deep`)](#deep-context-mode-deep)
+  - [Status Bar Context Tracking (`Ctx:` + `SNR:`)](#status-bar-context-tracking-ctx--snr)
+  - [Memex Experience Archive](#memex-experience-archive)
+  - [Design Rationale](#design-rationale)
+  - [Domain-Aware Preservation](#domain-aware-preservation)
 - [Personality Core — SAC Framework Style Control](#personality-core--sac-framework-style-control)
+  - [How It Works](#how-it-works-4)
+  - [What Changes Per Style](#what-changes-per-style)
+  - [Persistence](#persistence)
+  - [Research Provenance](#research-provenance-1)
 - [Emotion Engine — Affective State Modulation](#emotion-engine--affective-state-modulation)
+  - [Emotion Center (LLM-Generated Labels)](#emotion-center-llm-generated-labels)
+  - [TUI Status Bar](#tui-status-bar)
+  - [Proactive Admin Outreach](#proactive-admin-outreach)
+  - [Momentum Effects](#momentum-effects)
+  - [Research Foundations](#research-foundations)
 - [Voice Feedback (TTS)](#voice-feedback-tts)
+  - [LuxTTS Voice Cloning](#luxtts-voice-cloning)
+  - [Narration Engine Architecture](#narration-engine-architecture)
+  - [Emotion-Driven Prosody (SEST)](#emotion-driven-prosody-sest)
+  - [Personality-Aware Voice](#personality-aware-voice)
+  - [Voice Narration Research Foundations](#voice-narration-research-foundations)
+  - [Live Voice Session](#live-voice-session)
+  - [Telegram Voice Messages](#telegram-voice-messages)
+  - [Auto-Install Dependencies](#auto-install-dependencies)
+  - [Call Sub-Agent Architecture](#call-sub-agent-architecture)
+  - [Content-Aware Voice Narration](#content-aware-voice-narration)
 - [Listen Mode — Live Bidirectional Audio](#listen-mode--live-bidirectional-audio)
 - [Vision & Desktop Automation (Moondream)](#vision--desktop-automation-moondream)
+  - [Desktop Awareness](#desktop-awareness)
+  - [Vision Analysis](#vision-analysis)
+  - [Point-and-Click](#point-and-click)
+  - [Browser Automation](#browser-automation)
 - [Interactive TUI](#interactive-tui)
+  - [Slash Commands](#slash-commands)
+  - [Mid-Task Steering (Sub-Agent Architecture)](#mid-task-steering-sub-agent-architecture)
 - [Telegram Bridge — Sub-Agent Per Chat](#telegram-bridge--sub-agent-per-chat)
+  - [Admin Slash Command Passthrough](#admin-slash-command-passthrough)
+  - [Sub-Agent Architecture](#sub-agent-architecture)
+  - [Access Levels](#access-levels)
+  - [Streaming Responses](#streaming-responses)
+  - [Public User Isolation](#public-user-isolation)
+  - [Context-Aware Tool Policy](#context-aware-tool-policy)
+  - [Group Chat Distinction](#group-chat-distinction)
+  - [Media Handling](#media-handling)
+  - [Rate Limit Handling](#rate-limit-handling)
 - [x402 Payment Rails & Nexus P2P](#x402-payment-rails--nexus-p2p)
+  - [Wallet & Identity](#wallet--identity)
+  - [Expose Inference with Pricing](#expose-inference-with-pricing)
+  - [Spend — Gasless USDC Transfers (EIP-3009)](#spend--gasless-usdc-transfers-eip-3009)
+  - [Remote Inference — Tap Into the Mesh](#remote-inference--tap-into-the-mesh)
+  - [Ledger & Budget](#ledger--budget)
+  - [How x402 Works (End to End)](#how-x402-works-end-to-end)
+  - [Security Model](#security-model)
 - [Sponsored Inference — Share Your GPU With the World](#sponsored-inference--share-your-gpu-with-the-world)
+  - [For Sponsors: `/sponsor`](#for-sponsors-sponsor)
+  - [For Consumers: `/endpoint sponsor`](#for-consumers-endpoint-sponsor)
+  - [Architecture](#architecture-2)
+  - [Ollama Endpoint Security](#ollama-endpoint-security)
 - [COHERE Distributed Mind](#cohere-distributed-mind)
+  - [How COHERE Works](#how-cohere-works)
+  - [NATS Channels](#nats-channels)
+  - [Model Selection (Family-Based Scoring)](#model-selection-family-based-scoring)
+  - [Pressure Gate (CM-04)](#pressure-gate-cm-04)
 - [Self-Improvement & Learning](#self-improvement--learning)
+  - [Trajectory Logging](#trajectory-logging)
+  - [Rejection Fine-Tuning Pipeline](#rejection-fine-tuning-pipeline)
+  - [Inference-Time Self-Improvement](#inference-time-self-improvement)
 - [Dream Mode — Creative Idle Exploration](#dream-mode--creative-idle-exploration)
 - [Blessed Mode — Infinite Warm Loop](#blessed-mode--infinite-warm-loop)
+  - [Default Mode Network (DMN) — Autonomous Task Chaining](#default-mode-network-dmn--autonomous-task-chaining)
 - [Docker Sandbox & Collective Intelligence](#docker-sandbox--collective-intelligence)
+  - [Container Sandbox](#container-sandbox)
+  - [Multi-Agent Collective Testbed](#multi-agent-collective-testbed)
+  - [Self-Play Idle Loop (D1)](#self-play-idle-loop-d1)
+  - [Heuristic Extraction (D2)](#heuristic-extraction-d2)
+  - [Identity Kernel Evolution (D3)](#identity-kernel-evolution-d3)
+  - [Peer Delta Merge (D4)](#peer-delta-merge-d4)
+  - [6-Agent Evaluation Results](#6-agent-evaluation-results)
 - [Code Sandbox](#code-sandbox)
 - [Structured Data Tools](#structured-data-tools)
 - [On-Device Web Search](#on-device-web-search)
@@ -69,9 +199,19 @@ An autonomous multi-turn tool-calling agent that reads your code, makes changes,
 - [Cost Tracking & Session Metrics](#cost-tracking--session-metrics)
 - [Configuration](#configuration)
   - [Network Access & Binding](#network-access--binding)
+  - [Project Context](#project-context)
+  - [`.oa/` Project Directory](#oa-project-directory)
 - [Model Support](#model-support)
 - [Supported Inference Providers](#supported-inference-providers)
+  - [Connecting to a Provider](#connecting-to-a-provider)
+  - [P2P Inference via libp2p](#p2p-inference-via-libp2p)
+  - [Endpoint Cascade Failover](#endpoint-cascade-failover)
 - [Evaluation Suite](#evaluation-suite)
+  - [Benchmark Results](#benchmark-results)
+  - [Collective Intelligence Evaluation (v0.186.57)](#collective-intelligence-evaluation-v018657)
+  - [Web Navigation Evaluation (v0.186.61)](#web-navigation-evaluation-v018661)
+  - [Multi-Agent Architecture Evaluation (v0.187.4)](#multi-agent-architecture-evaluation-v01874)
+  - [REST API Enterprise Evaluation (v0.185.68)](#rest-api-enterprise-evaluation-v018568)
 - [AIWG Integration](#aiwg-integration)
 - [Research Citations](#research-citations)
 - [License](#license)
@@ -310,6 +450,14 @@ OA_API_KEYS="key1:admin:alice:30:50000:5,key2:run:ci:60::3,key3:read:grafana" oa
 
 > **Every example below is verified against `open-agents-ai@0.187.189` on a live daemon.** Examples from earlier versions are deprecated.
 
+#### Access Policy & Binding
+
+Control who can reach the daemon and where it binds:
+
+- TUI commands: `/access loopback|lan|any`, `/host <host[:port]>`, `/network config` (interactive), `--local` to save per‑project.
+- Environment: `OA_ACCESS=loopback|lan|any`, `OA_HOST=host[:port]`.
+- See Configuration → [Network Access & Binding](#network-access--binding) for full details and security guidance.
+
 #### Working Directory
 
 Pass `X-Working-Directory` header to run commands in your current terminal directory:
@@ -3523,6 +3671,18 @@ Post-task, the [ReadAgent](https://arxiv.org/abs/2402.09727) gist compressor cre
 
 ### Near-Critical Cognitive Architecture
 
+### Cross‑Modality Identity & Association (CLIP + Voice)
+
+Open Agents binds entities across image, audio, and text using joint‑embedding models:
+
+- CLIP‑based visual ID: person/object embeddings extracted from frames are matched to persistent entity nodes; cosine similarity > τ promotes to identity with temporal smoothing. Supports multi‑appearance tracking and re‑identification across sessions.
+- Voiceprint linkage: speaker embeddings (x‑vector/ECAPA) are associated with entities when co‑occurring in time with a visual track and a transcribed utterance; robust to background noise via median pooling across windows.
+- Text label fusion: natural‑language labels (names, roles, tags) are bound to the same entity when co‑referents appear in proximate context windows (heuristics + clustering).
+- Association graph: cross‑modal edges (image↔voice↔text) consolidate into a unified entity node with provenance (model, score, timestamp) and decay‑based confidence.
+- Privacy & safety: raw media never leaves the machine; embeddings are stored locally under `.oa/memory/`. Redaction controls can drop embeddings by label or recency.
+
+This enables queries like: “Find where Alex spoke about deployment,” “Show files edited after the person in the red sweater approved the PR,” or “Summarize conversations where Speaker‑B and Alice appear together.”
+
 The associative memory integrates with a near-critical cognitive framework inspired by [Beggs & Plenz (2003)](https://doi.org/10.1523/JNEUROSCI.23-35-11167.2003) neuronal avalanche dynamics:
 
 - **Auto-consolidation**: At task boundaries, the system writes consolidation snapshots to `.oa/consolidations/` with lessons learned and key patterns
@@ -4507,4 +4667,3 @@ Security tips:
 - Prefer `loopback` on single-machine setups.
 - Use `lan` for same-network devices.
 - If you set `any`, put the daemon behind TLS and an authenticating reverse proxy.
-