open-agents-ai 0.46.0 → 0.48.0

package/README.md

@@ -39,6 +39,8 @@ An autonomous multi-turn tool-calling agent that reads your code, makes changes,
 - **Dream Mode** — creative idle exploration modeled after real sleep architecture (NREM→REM cycles)
 - **Autoresearch Swarm** — 5-agent GPU experiment loop during REM sleep: Researcher, Monitor, Evaluator, Critic, Flow Maintainer autonomously run ML training experiments, keep improvements, discard regressions
 - **Live Listen** — bidirectional voice communication with real-time Whisper transcription
+- **Live Voice Session** — `/listen` with `/voice` enabled spawns a cloudflared tunnel with a real-time WebSocket audio endpoint. A floating presence UI shows live transcription, connected users, and audio visualization. Echo cancellation prevents TTS feedback loops
+- **Telegram Voice** — `/voice` enabled via Telegram forwards TTS audio as voice messages alongside text responses. Incoming voice messages are auto-transcribed and handled as text
 - **Neural TTS** — hear what the agent is doing via GLaDOS or Overwatch ONNX voices, with personality-driven expressiveness
 - **Personality Core** — SAC framework-based style control (concise/balanced/verbose/pedagogical) that shapes agent response depth, voice expressiveness, and system prompt behavior
 - **Human expert speed ratio** — real-time `Exp: Nx` gauge comparing agent speed to a leading human expert, calibrated across 47 tool baselines
@@ -333,6 +335,12 @@ Memory flows bidirectionally: the swarm reads all 5 keys at startup (Phase 0) an
 
 The Monitor agent can be "detached" between experiment rounds by the Flow Maintainer. When detached, the monitor receives a sub-task (e.g., "analyze GPU memory patterns from last 3 runs") instead of its standard watch prompt. This lets the swarm use idle monitoring capacity for useful analysis work.
 
+#### Dependency Management
+
+The autoresearch tool uses [`uv`](https://docs.astral.sh/uv/) for zero-setup Python environment management. Running `autoresearch(action="setup")` creates a `pyproject.toml` with all dependencies (torch, kernels, pyarrow, rustbpe, tiktoken, etc.) and runs `uv sync` to create a `.venv` automatically.
+
+If the Python scripts are invoked directly (without `uv run`), they self-bootstrap: detect missing packages, create a local `.venv`, install dependencies (including CUDA 12.8 torch), and re-exec with the venv's Python. This handles cases where the agent calls `python3 prepare.py` instead of `uv run prepare.py`.
+
 If no GPU is detected, the REM stage falls back to the standard multi-agent creative exploration (Visionary + Pragmatist + Cross-Pollinator + Synthesizer).
 
 ## Blessed Mode — Infinite Warm Loop
@@ -386,6 +394,23 @@ Connect the agent to a Telegram bot. Each incoming message spawns a dedicated su
 
 The bot token and admin ID are persisted to project settings, so you only need to set them once. After that, bare `/telegram` toggles the bridge on and off like a service watchdog.
 
+### Admin Slash Command Passthrough
+
+When the admin sends a `/command` in a private DM, it's routed directly through the terminal's command handler — the same code path as typing the command in the TUI. This means you can control the agent from your phone:
+
+```
+/model qwen3.5:122b     → switch model
+/voice                   → toggle TTS
+/dream                   → enter dream mode
+/listen                  → toggle voice input
+/stats                   → show session metrics
+/config                  → show current config
+/bless                   → toggle blessed mode
+/telegram status         → check bridge status
+```
+
+The command output is captured, ANSI-stripped, and sent back as a Telegram message. Skill invocations (e.g., `/ralph`, `/eval-agent`) are queued as tasks.
+
 ### Sub-Agent Architecture
 
 Each Telegram message spawns an independent `AgenticRunner` sub-agent. Sub-agent tool calls, status updates, and streaming tokens appear in the terminal waterfall view with `✈ @username` prefixes — so you can watch all Telegram conversations happening alongside your main work.
@@ -982,6 +1007,50 @@ Voice output adapts to the active personality style — the same tool call sound
 
 Task completion, tool failures, and all TTS announcements follow the same personality tier. Set the style with `/style verbose` and the voice output becomes conversational rather than robotic.
 
+### Live Voice Session
+
+When both `/voice` and `/listen` are enabled, the system spawns a **live voice session** — a real-time bidirectional audio endpoint exposed through a cloudflared tunnel:
+
+```bash
+/voice              # Enable TTS
+/listen             # Starts mic + spawns voice session
+```
+
+What happens:
+1. A local HTTP + WebSocket server starts on a random port
+2. `cloudflared tunnel --url` exposes it publicly with a `*.trycloudflare.com` URL
+3. The terminal shows a `☁` cloud icon with live session runtime
+4. Visiting the URL shows a **floating presence** UI that:
+   - Undulates with the model's TTS audio output
+   - Captures your microphone (with echo cancellation)
+   - Shows live transcription for both sides
+   - Displays connected users
+
+**Echo cancellation**: The server mutes ASR input while TTS is playing, preventing the model from hearing its own voice.
+
+**Terminal waterfall**: The cloud session sits in the normal TUI waterfall alongside other activity, showing connected users and session runtime.
+
+```
+  ☁ Live Voice Session
+    ⎿ URL: https://abc-xyz.trycloudflare.com
+    ⎿ Bidirectional PCM audio + live transcription
+    ⎿ → web-user connected
+    ⎿ ☁ [user] hello, what are you working on?
+    ⎿ ☁ [agent] I'm analyzing the codebase structure...
+```
+
+Stop with `/listen stop` or `/listen off`.
+
+### Telegram Voice Messages
+
+When `/voice` is enabled and the Telegram bridge is active:
+- **Outgoing**: Agent responses are synthesized to audio via TTS and sent as Telegram voice messages (OGG/Opus) alongside the text response
+- **Incoming**: Voice messages sent to the bot are auto-transcribed via Whisper and handled as text — no need for the agent to explicitly call `transcribe_file`
+
+### Auto-Install Dependencies
+
+Cloudflared is automatically installed at startup alongside other dependencies (moondream, tesseract, transcribe-cli). The install is non-blocking and runs in the background.
+
 ## Personality Core — SAC Framework Style Control
 
 The personality system controls how the agent communicates — from silent operator to teacher mode. It's based on the **SAC framework** (arXiv:2506.20993) which models personality along five behavioral intensity dimensions rather than binary trait toggles.