durar-ai 2026.4.4

package/docs/tts.md

@@ -0,0 +1,452 @@
+---
+summary: "Text-to-speech (TTS) for outbound replies"
+read_when:
+  - Enabling text-to-speech for replies
+  - Configuring TTS providers or limits
+  - Using /tts commands
+title: "Text-to-Speech (legacy path)"
+---
+
+# Text-to-speech (TTS)
+
+Durar can convert outbound replies into audio using ElevenLabs, Microsoft, MiniMax, or OpenAI.
+It works anywhere Durar can send audio.
+
+## Supported services
+
+- **ElevenLabs** (primary or fallback provider)
+- **Microsoft** (primary or fallback provider; current bundled implementation uses `node-edge-tts`)
+- **MiniMax** (primary or fallback provider; uses the T2A v2 API)
+- **OpenAI** (primary or fallback provider; also used for summaries)
+
+### Microsoft speech notes
+
+The bundled Microsoft speech provider currently uses Microsoft Edge's online
+neural TTS service via the `node-edge-tts` library. It's a hosted service (not
+local), uses Microsoft endpoints, and does not require an API key.
+`node-edge-tts` exposes speech configuration options and output formats, but
+not all options are supported by the service. Legacy config and directive input
+using `edge` still works and is normalized to `microsoft`.
+
+Because this path is a public web service without a published SLA or quota,
+treat it as best-effort. If you need guaranteed limits and support, use OpenAI
+or ElevenLabs.
+
+## Optional keys
+
+If you want OpenAI, ElevenLabs, or MiniMax:
+
+- `ELEVENLABS_API_KEY` (or `XI_API_KEY`)
+- `MINIMAX_API_KEY`
+- `OPENAI_API_KEY`
+
+Microsoft speech does **not** require an API key.
+
+If multiple providers are configured, the selected provider is used first and the others are fallback options.
+Auto-summary uses the configured `summaryModel` (or `agents.defaults.model.primary`),
+so that provider must also be authenticated if you enable summaries.
+
+## Service links
+
+- [OpenAI Text-to-Speech guide](https://platform.openai.com/docs/guides/text-to-speech)
+- [OpenAI Audio API reference](https://platform.openai.com/docs/api-reference/audio)
+- [ElevenLabs Text to Speech](https://elevenlabs.io/docs/api-reference/text-to-speech)
+- [ElevenLabs Authentication](https://elevenlabs.io/docs/api-reference/authentication)
+- [MiniMax T2A v2 API](https://platform.minimaxi.com/document/T2A%20V2)
+- [node-edge-tts](https://github.com/SchneeHertz/node-edge-tts)
+- [Microsoft Speech output formats](https://learn.microsoft.com/azure/ai-services/speech-service/rest-text-to-speech#audio-outputs)
+
+## Is it enabled by default?
+
+No. Auto‑TTS is **off** by default. Enable it in config with
+`messages.tts.auto` or per session with `/tts always` (alias: `/tts on`).
+
+When `messages.tts.provider` is unset, Durar picks the first configured
+speech provider in registry auto-select order.
+
+## Config
+
+TTS config lives under `messages.tts` in `Durar.json`.
+Full schema is in [Gateway configuration](/gateway/configuration).
+
+### Minimal config (enable + provider)
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "always",
+      provider: "elevenlabs",
+    },
+  },
+}
+```
+
+### OpenAI primary with ElevenLabs fallback
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "always",
+      provider: "openai",
+      summaryModel: "openai/gpt-4.1-mini",
+      modelOverrides: {
+        enabled: true,
+      },
+      providers: {
+        openai: {
+          apiKey: "openai_api_key",
+          baseUrl: "https://api.openai.com/v1",
+          model: "gpt-4o-mini-tts",
+          voice: "alloy",
+        },
+        elevenlabs: {
+          apiKey: "elevenlabs_api_key",
+          baseUrl: "https://api.elevenlabs.io",
+          voiceId: "voice_id",
+          modelId: "eleven_multilingual_v2",
+          seed: 42,
+          applyTextNormalization: "auto",
+          languageCode: "en",
+          voiceSettings: {
+            stability: 0.5,
+            similarityBoost: 0.75,
+            style: 0.0,
+            useSpeakerBoost: true,
+            speed: 1.0,
+          },
+        },
+      },
+    },
+  },
+}
+```
+
+### Microsoft primary (no API key)
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "always",
+      provider: "microsoft",
+      providers: {
+        microsoft: {
+          enabled: true,
+          voice: "en-US-MichelleNeural",
+          lang: "en-US",
+          outputFormat: "audio-24khz-48kbitrate-mono-mp3",
+          rate: "+10%",
+          pitch: "-5%",
+        },
+      },
+    },
+  },
+}
+```
+
+### MiniMax primary
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "always",
+      provider: "minimax",
+      providers: {
+        minimax: {
+          apiKey: "minimax_api_key",
+          baseUrl: "https://api.minimax.io",
+          model: "speech-2.8-hd",
+          voiceId: "English_expressive_narrator",
+          speed: 1.0,
+          vol: 1.0,
+          pitch: 0,
+        },
+      },
+    },
+  },
+}
+```
+
+### Disable Microsoft speech
+
+```json5
+{
+  messages: {
+    tts: {
+      providers: {
+        microsoft: {
+          enabled: false,
+        },
+      },
+    },
+  },
+}
+```
+
+### Custom limits + prefs path
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "always",
+      maxTextLength: 4000,
+      timeoutMs: 30000,
+      prefsPath: "~/.Durar/settings/tts.json",
+    },
+  },
+}
+```
+
+### Only reply with audio after an inbound voice message
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "inbound",
+    },
+  },
+}
+```
+
+### Disable auto-summary for long replies
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "always",
+    },
+  },
+}
+```
+
+Then run:
+
+```
+/tts summary off
+```
+
+### Notes on fields
+
+- `auto`: auto‑TTS mode (`off`, `always`, `inbound`, `tagged`).
+  - `inbound` only sends audio after an inbound voice message.
+  - `tagged` only sends audio when the reply includes `[[tts]]` tags.
+- `enabled`: legacy toggle (doctor migrates this to `auto`).
+- `mode`: `"final"` (default) or `"all"` (includes tool/block replies).
+- `provider`: speech provider id such as `"elevenlabs"`, `"microsoft"`, `"minimax"`, or `"openai"` (fallback is automatic).
+- If `provider` is **unset**, Durar uses the first configured speech provider in registry auto-select order.
+- Legacy `provider: "edge"` still works and is normalized to `microsoft`.
+- `summaryModel`: optional cheap model for auto-summary; defaults to `agents.defaults.model.primary`.
+  - Accepts `provider/model` or a configured model alias.
+- `modelOverrides`: allow the model to emit TTS directives (on by default).
+  - `allowProvider` defaults to `false` (provider switching is opt-in).
+- `providers.<id>`: provider-owned settings keyed by speech provider id.
+- Legacy direct provider blocks (`messages.tts.openai`, `messages.tts.elevenlabs`, `messages.tts.microsoft`, `messages.tts.edge`) are auto-migrated to `messages.tts.providers.<id>` on load.
+- `maxTextLength`: hard cap for TTS input (chars). `/tts audio` fails if exceeded.
+- `timeoutMs`: request timeout (ms).
+- `prefsPath`: override the local prefs JSON path (provider/limit/summary).
+- `apiKey` values fall back to env vars (`ELEVENLABS_API_KEY`/`XI_API_KEY`, `MINIMAX_API_KEY`, `OPENAI_API_KEY`).
+- `providers.elevenlabs.baseUrl`: override ElevenLabs API base URL.
+- `providers.openai.baseUrl`: override the OpenAI TTS endpoint.
+  - Resolution order: `messages.tts.providers.openai.baseUrl` -> `OPENAI_TTS_BASE_URL` -> `https://api.openai.com/v1`
+  - Non-default values are treated as OpenAI-compatible TTS endpoints, so custom model and voice names are accepted.
+- `providers.elevenlabs.voiceSettings`:
+  - `stability`, `similarityBoost`, `style`: `0..1`
+  - `useSpeakerBoost`: `true|false`
+  - `speed`: `0.5..2.0` (1.0 = normal)
+- `providers.elevenlabs.applyTextNormalization`: `auto|on|off`
+- `providers.elevenlabs.languageCode`: 2-letter ISO 639-1 (e.g. `en`, `de`)
+- `providers.elevenlabs.seed`: integer `0..4294967295` (best-effort determinism)
+- `providers.minimax.baseUrl`: override MiniMax API base URL (default `https://api.minimax.io`, env: `MINIMAX_API_HOST`).
+- `providers.minimax.model`: TTS model (default `speech-2.8-hd`, env: `MINIMAX_TTS_MODEL`).
+- `providers.minimax.voiceId`: voice identifier (default `English_expressive_narrator`, env: `MINIMAX_TTS_VOICE_ID`).
+- `providers.minimax.speed`: playback speed `0.5..2.0` (default 1.0).
+- `providers.minimax.vol`: volume `(0, 10]` (default 1.0; must be greater than 0).
+- `providers.minimax.pitch`: pitch shift `-12..12` (default 0).
+- `providers.microsoft.enabled`: allow Microsoft speech usage (default `true`; no API key).
+- `providers.microsoft.voice`: Microsoft neural voice name (e.g. `en-US-MichelleNeural`).
+- `providers.microsoft.lang`: language code (e.g. `en-US`).
+- `providers.microsoft.outputFormat`: Microsoft output format (e.g. `audio-24khz-48kbitrate-mono-mp3`).
+  - See Microsoft Speech output formats for valid values; not all formats are supported by the bundled Edge-backed transport.
+- `providers.microsoft.rate` / `providers.microsoft.pitch` / `providers.microsoft.volume`: percent strings (e.g. `+10%`, `-5%`).
+- `providers.microsoft.saveSubtitles`: write JSON subtitles alongside the audio file.
+- `providers.microsoft.proxy`: proxy URL for Microsoft speech requests.
+- `providers.microsoft.timeoutMs`: request timeout override (ms).
+- `edge.*`: legacy alias for the same Microsoft settings.
+
+## Model-driven overrides (default on)
+
+By default, the model **can** emit TTS directives for a single reply.
+When `messages.tts.auto` is `tagged`, these directives are required to trigger audio.
+
+When enabled, the model can emit `[[tts:...]]` directives to override the voice
+for a single reply, plus an optional `[[tts:text]]...[[/tts:text]]` block to
+provide expressive tags (laughter, singing cues, etc) that should only appear in
+the audio.
+
+`provider=...` directives are ignored unless `modelOverrides.allowProvider: true`.
+
+Example reply payload:
+
+```
+Here you go.
+
+[[tts:voiceId=pMsXgVXv3BLzUgSXRplE model=eleven_v3 speed=1.1]]
+[[tts:text]](laughs) Read the song once more.[[/tts:text]]
+```
+
+Available directive keys (when enabled):
+
+- `provider` (registered speech provider id, for example `openai`, `elevenlabs`, `minimax`, or `microsoft`; requires `allowProvider: true`)
+- `voice` (OpenAI voice) or `voiceId` (ElevenLabs / MiniMax)
+- `model` (OpenAI TTS model, ElevenLabs model id, or MiniMax model)
+- `stability`, `similarityBoost`, `style`, `speed`, `useSpeakerBoost`
+- `vol` / `volume` (MiniMax volume, 0-10)
+- `pitch` (MiniMax pitch, -12 to 12)
+- `applyTextNormalization` (`auto|on|off`)
+- `languageCode` (ISO 639-1)
+- `seed`
+
+Disable all model overrides:
+
+```json5
+{
+  messages: {
+    tts: {
+      modelOverrides: {
+        enabled: false,
+      },
+    },
+  },
+}
+```
+
+Optional allowlist (enable provider switching while keeping other knobs configurable):
+
+```json5
+{
+  messages: {
+    tts: {
+      modelOverrides: {
+        enabled: true,
+        allowProvider: true,
+        allowSeed: false,
+      },
+    },
+  },
+}
+```
+
+## Per-user preferences
+
+Slash commands write local overrides to `prefsPath` (default:
+`~/.Durar/settings/tts.json`, override with `Durar_TTS_PREFS` or
+`messages.tts.prefsPath`).
+
+Stored fields:
+
+- `enabled`
+- `provider`
+- `maxLength` (summary threshold; default 1500 chars)
+- `summarize` (default `true`)
+
+These override `messages.tts.*` for that host.
+
+## Output formats (fixed)
+
+- **Feishu / Matrix / Telegram / WhatsApp**: Opus voice message (`opus_48000_64` from ElevenLabs, `opus` from OpenAI).
+  - 48kHz / 64kbps is a good voice message tradeoff.
+- **Other channels**: MP3 (`mp3_44100_128` from ElevenLabs, `mp3` from OpenAI).
+  - 44.1kHz / 128kbps is the default balance for speech clarity.
+- **MiniMax**: MP3 (`speech-2.8-hd` model, 32kHz sample rate). Voice-note format not natively supported; use OpenAI or ElevenLabs for guaranteed Opus voice messages.
+- **Microsoft**: uses `microsoft.outputFormat` (default `audio-24khz-48kbitrate-mono-mp3`).
+  - The bundled transport accepts an `outputFormat`, but not all formats are available from the service.
+  - Output format values follow Microsoft Speech output formats (including Ogg/WebM Opus).
+  - Telegram `sendVoice` accepts OGG/MP3/M4A; use OpenAI/ElevenLabs if you need
+    guaranteed Opus voice messages.
+  - If the configured Microsoft output format fails, Durar retries with MP3.
+
+OpenAI/ElevenLabs output formats are fixed per channel (see above).
+
+## Auto-TTS behavior
+
+When enabled, Durar:
+
+- skips TTS if the reply already contains media or a `MEDIA:` directive.
+- skips very short replies (< 10 chars).
+- summarizes long replies when enabled using `agents.defaults.model.primary` (or `summaryModel`).
+- attaches the generated audio to the reply.
+
+If the reply exceeds `maxLength` and summary is off (or no API key for the
+summary model), audio
+is skipped and the normal text reply is sent.
+
+## Flow diagram
+
+```
+Reply -> TTS enabled?
+  no  -> send text
+  yes -> has media / MEDIA: / short?
+          yes -> send text
+          no  -> length > limit?
+                   no  -> TTS -> attach audio
+                   yes -> summary enabled?
+                            no  -> send text
+                            yes -> summarize (summaryModel or agents.defaults.model.primary)
+                                      -> TTS -> attach audio
+```
+
+## Slash command usage
+
+There is a single command: `/tts`.
+See [Slash commands](/tools/slash-commands) for enablement details.
+
+Discord note: `/tts` is a built-in Discord command, so Durar registers
+`/voice` as the native command there. Text `/tts ...` still works.
+
+```
+/tts off
+/tts always
+/tts inbound
+/tts tagged
+/tts status
+/tts provider openai
+/tts limit 2000
+/tts summary off
+/tts audio Hello from Durar
+```
+
+Notes:
+
+- Commands require an authorized sender (allowlist/owner rules still apply).
+- `commands.text` or native command registration must be enabled.
+- `off|always|inbound|tagged` are per‑session toggles (`/tts on` is an alias for `/tts always`).
+- `limit` and `summary` are stored in local prefs, not the main config.
+- `/tts audio` generates a one-off audio reply (does not toggle TTS on).
+- `/tts status` includes fallback visibility for the latest attempt:
+  - success fallback: `Fallback: <primary> -> <used>` plus `Attempts: ...`
+  - failure: `Error: ...` plus `Attempts: ...`
+  - detailed diagnostics: `Attempt details: provider:outcome(reasonCode) latency`
+- OpenAI and ElevenLabs API failures now include parsed provider error detail and request id (when returned by the provider), which is surfaced in TTS errors/logs.
+
+## Agent tool
+
+The `tts` tool converts text to speech and returns an audio attachment for
+reply delivery. When the channel is Feishu, Matrix, Telegram, or WhatsApp,
+the audio is delivered as a voice message rather than a file attachment.
+
+## Gateway RPC
+
+Gateway methods:
+
+- `tts.status`
+- `tts.enable`
+- `tts.disable`
+- `tts.convert`
+- `tts.setProvider`
+- `tts.providers`

package/docs/vps.md

@@ -0,0 +1,115 @@
+---
+summary: "Run Durar on a Linux server or cloud VPS — provider picker, architecture, and tuning"
+read_when:
+  - You want to run the Gateway on a Linux server or cloud VPS
+  - You need a quick map of hosting guides
+  - You want generic Linux server tuning for Durar
+title: "Linux Server"
+sidebarTitle: "Linux Server"
+---
+
+# Linux Server
+
+Run the Durar Gateway on any Linux server or cloud VPS. This page helps you
+pick a provider, explains how cloud deployments work, and covers generic Linux
+tuning that applies everywhere.
+
+## Pick a provider
+
+<CardGroup cols={2}>
+  <Card title="Railway" href="/install/railway">One-click, browser setup</Card>
+  <Card title="Northflank" href="/install/northflank">One-click, browser setup</Card>
+  <Card title="DigitalOcean" href="/install/digitalocean">Simple paid VPS</Card>
+  <Card title="Oracle Cloud" href="/install/oracle">Always Free ARM tier</Card>
+  <Card title="Fly.io" href="/install/fly">Fly Machines</Card>
+  <Card title="Hetzner" href="/install/hetzner">Docker on Hetzner VPS</Card>
+  <Card title="GCP" href="/install/gcp">Compute Engine</Card>
+  <Card title="Azure" href="/install/azure">Linux VM</Card>
+  <Card title="exe.dev" href="/install/exe-dev">VM with HTTPS proxy</Card>
+  <Card title="Raspberry Pi" href="/install/raspberry-pi">ARM self-hosted</Card>
+</CardGroup>
+
+**AWS (EC2 / Lightsail / free tier)** also works well.
+A community video walkthrough is available at
+[x.com/techfrenAJ/status/2014934471095812547](https://x.com/techfrenAJ/status/2014934471095812547)
+(community resource -- may become unavailable).
+
+## How cloud setups work
+
+- The **Gateway runs on the VPS** and owns state + workspace.
+- You connect from your laptop or phone via the **Control UI** or **Tailscale/SSH**.
+- Treat the VPS as the source of truth and **back up** the state + workspace regularly.
+- Secure default: keep the Gateway on loopback and access it via SSH tunnel or Tailscale Serve.
+  If you bind to `lan` or `tailnet`, require `gateway.auth.token` or `gateway.auth.password`.
+
+Related pages: [Gateway remote access](/gateway/remote), [Platforms hub](/platforms).
+
+## Shared company agent on a VPS
+
+Running a single agent for a team is a valid setup when every user is in the same trust boundary and the agent is business-only.
+
+- Keep it on a dedicated runtime (VPS/VM/container + dedicated OS user/accounts).
+- Do not sign that runtime into personal Apple/Google accounts or personal browser/password-manager profiles.
+- If users are adversarial to each other, split by gateway/host/OS user.
+
+Security model details: [Security](/gateway/security).
+
+## Using nodes with a VPS
+
+You can keep the Gateway in the cloud and pair **nodes** on your local devices
+(Mac/iOS/Android/headless). Nodes provide local screen/camera/canvas and `system.run`
+capabilities while the Gateway stays in the cloud.
+
+Docs: [Nodes](/nodes), [Nodes CLI](/cli/nodes).
+
+## Startup tuning for small VMs and ARM hosts
+
+If CLI commands feel slow on low-power VMs (or ARM hosts), enable Node's module compile cache:
+
+```bash
+grep -q 'NODE_COMPILE_CACHE=/var/tmp/Durar-compile-cache' ~/.bashrc || cat >> ~/.bashrc <<'EOF'
+export NODE_COMPILE_CACHE=/var/tmp/Durar-compile-cache
+mkdir -p /var/tmp/Durar-compile-cache
+export Durar_NO_RESPAWN=1
+EOF
+source ~/.bashrc
+```
+
+- `NODE_COMPILE_CACHE` improves repeated command startup times.
+- `Durar_NO_RESPAWN=1` avoids extra startup overhead from a self-respawn path.
+- First command run warms the cache; subsequent runs are faster.
+- For Raspberry Pi specifics, see [Raspberry Pi](/install/raspberry-pi).
+
+### systemd tuning checklist (optional)
+
+For VM hosts using `systemd`, consider:
+
+- Add service env for a stable startup path:
+  - `Durar_NO_RESPAWN=1`
+  - `NODE_COMPILE_CACHE=/var/tmp/Durar-compile-cache`
+- Keep restart behavior explicit:
+  - `Restart=always`
+  - `RestartSec=2`
+  - `TimeoutStartSec=90`
+- Prefer SSD-backed disks for state/cache paths to reduce random-I/O cold-start penalties.
+
+For the standard `Durar onboard --install-daemon` path, edit the user unit:
+
+```bash
+systemctl --user edit Durar-gateway.service
+```
+
+```ini
+[Service]
+Environment=Durar_NO_RESPAWN=1
+Environment=NODE_COMPILE_CACHE=/var/tmp/Durar-compile-cache
+Restart=always
+RestartSec=2
+TimeoutStartSec=90
+```
+
+If you deliberately installed a system unit instead, edit
+`Durar-gateway.service` via `sudo systemctl edit Durar-gateway.service`.
+
+How `Restart=` policies help automated recovery:
+[systemd can automate service recovery](https://www.redhat.com/en/blog/systemd-automate-recovery).