@llblab/pi-telegram 0.10.7 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +10 -6
- package/BACKLOG.md +5 -0
- package/CHANGELOG.md +26 -1
- package/README.md +48 -24
- package/docs/README.md +3 -2
- package/docs/architecture.md +3 -3
- package/docs/extension-sections.md +6 -4
- package/docs/inbound-handlers.md +41 -1
- package/docs/outbound-handlers.md +32 -49
- package/docs/voice.md +210 -0
- package/index.ts +77 -1
- package/lib/api.ts +35 -2
- package/lib/commands.ts +18 -0
- package/lib/config.ts +70 -0
- package/lib/extension-sections.ts +39 -20
- package/lib/external-handlers.ts +3 -4
- package/lib/inbound-handlers.ts +197 -0
- package/lib/media.ts +3 -0
- package/lib/menu-settings.ts +134 -8
- package/lib/menu-status.ts +17 -3
- package/lib/menu-thinking.ts +12 -1
- package/lib/menu.ts +10 -1
- package/lib/outbound-handlers.ts +719 -277
- package/lib/preview.ts +9 -0
- package/lib/prompts.ts +2 -1
- package/lib/queue.ts +84 -5
- package/lib/routing.ts +14 -1
- package/lib/turns.ts +84 -20
- package/lib/voice.ts +295 -0
- package/package.json +1 -1
package/AGENTS.md
CHANGED
|
@@ -57,6 +57,7 @@
|
|
|
57
57
|
- The bridge is session-local, paired to one allowed Telegram user, and owns a local queue aligned with π lifecycle hooks
|
|
58
58
|
- Queue admission is explicit and validated: immediate commands, control lane, priority lane, and default lane must preserve allowed kind/lane pairings
|
|
59
59
|
- Dispatch is gated by active turns, pending dispatch, unsettled control work, compaction, `ctx.isIdle()`, and π pending messages; dispatched prompts remain queued until `agent_start` consumes them
|
|
60
|
+
- Telegram `/compact` owns a native `typing` keepalive for the compaction window so phone clients show activity between the started/completed notices; stop it on both completion and failure
|
|
60
61
|
- `/stop`, `/abort`, `/next`, and `/continue` have distinct contracts: reset queue and abort; abort while preserving queue; force next queued turn; enqueue a priority `continue` prompt
|
|
61
62
|
- `/start`, `/help`, and `/status` open the unified command-help/status-row/control menu; `/model`, `/thinking`, and `/queue` jump to sections directly; visible bot commands are `/start`, `/compact`, `/next`, `/continue`, `/abort`, `/stop`
|
|
62
63
|
- Command/menu emoji are fixed UI adornments owned by the `commands` map; do not add a persisted emoji toggle or Settings menu until there is a real setting to own
|
|
@@ -70,7 +71,10 @@
|
|
|
70
71
|
- Real code blocks must stay literal and escaped
|
|
71
72
|
- `telegram_attach` is the canonical outbound file-delivery path for Telegram-originated requests
|
|
72
73
|
- Telegram delivery strips top-level HTML comments from preview/final text; column-zero top-level `<!-- telegram_voice ... -->` and `<!-- telegram_button ... -->` blocks are special outbound comments handled after `agent_end` without requiring agent-side transport tool calls, while comments inside code, quotes, lists, or indented examples stay literal
|
|
73
|
-
- `telegram_voice` and `telegram_button` are not π tools; keep prompts/docs explicit that agents should author markup while
|
|
74
|
+
- `telegram_voice` and `telegram_button` are not π tools; keep prompts/docs explicit that agents should author markup while voice synthesis provider extensions own TTS/OGG conversion, and pi-telegram owns button routing plus Telegram delivery
|
|
75
|
+
- Voice reply policy and prompt context are owned by pi-telegram's `telegram.json` `voice.replyMode`: missing/invalid config behaves as `manual` but does not add a `[voice]` prompt-context block; only an explicit valid `voice.replyMode` renders context. Render a single voice field as `[voice] reply mode: manual|mirror|always`, and render multiple fields as a `[voice]` list; place voice context after `[outputs]` when handler output exists, otherwise after `[attachments]`; provider prompt contributions are optional provider-specific additions, not the default policy channel
|
|
76
|
+
- Voice reply mode Settings UI standard: the top-level Settings row is `👄 Voice reply: hidden|manual|mirror|always`; `hidden` is the true default and means no valid `voice.replyMode` is persisted, behavior is manual, and no voice policy is added to prompt context; explicit `manual` behaves the same operationally but renders reply-mode context. The submenu title is `Voice reply mode:`; choice buttons use lowercase labels with a model-style active dot (`🟢 hidden`, `🟢 mirror`) rather than per-mode emoji; the explanatory submenu body uses compact HTML-code bullets such as `<code>-</code> <code>hidden</code> (default): ...`. Preserve this wording/icons unless the operator explicitly asks to redesign it
|
|
77
|
+
- Outbound voice delivery is one fallback pipeline: configured `outboundHandlers` with `type: "voice"` run first in `telegram.json` order, then programmatic voice handlers, then registered voice synthesis providers as zero-config progressive fallbacks; provider extensions must not override operator-configured handlers
|
|
74
78
|
- `telegram_voice` text is arbitrary TTS-target text: use body form for multiline text, `<!-- telegram_voice text="Short summary" -->` for explicit one-line text, or `<!-- telegram_voice: Short summary -->` for one-line text with no attributes
|
|
75
79
|
- `telegram_button` has three canonical forms: `<!-- telegram_button: OK -->` for label-only buttons, `<!-- telegram_button label=Continue prompt="Continue with the current plan." -->` for one-line prompts, or `<!-- telegram_button label="Show risks"\nList the main risks first.\n-->` for multiline prompts
|
|
76
80
|
|
|
@@ -102,9 +106,9 @@ The canonical detailed ownership map lives in [`docs/architecture.md`](./docs/ar
|
|
|
102
106
|
|
|
103
107
|
- Scheduling and lifecycle: `queue`, `runtime`, `lifecycle`, `locks`
|
|
104
108
|
- Telegram transport and inbound flow: `api`, `polling`, `updates`, `routing`, `media`, `turns`, `inbound-handlers`, `config`, `setup`
|
|
105
|
-
- Response surfaces: `preview`, `replies`, `rendering`, `keyboard`, `outbound-attachments`, `outbound-handlers`, `status`
|
|
109
|
+
- Response surfaces: `preview`, `replies`, `rendering`, `keyboard`, `outbound-attachments`, `outbound-handlers`, `voice`, `status`
|
|
106
110
|
- Controls and application menu UI: `commands`, `menu`, `menu-model`, `menu-thinking`, `menu-status`, `menu-queue`, `model`, `prompts`
|
|
107
|
-
- Extension platform: `extension-sections` owns section registry, token mapping, callback dispatch, context building, and globalThis bridge
|
|
111
|
+
- Extension platform: `extension-sections` owns section registry, token mapping, callback dispatch, context building, and its globalThis bridge; `voice` owns the voice-provider registry and its globalThis bridge
|
|
108
112
|
- Pi SDK boundary: `pi` owns direct pi imports and bound extension API ports
|
|
109
113
|
|
|
110
114
|
## 6.4 Entrypoint And Import Boundaries
|
|
@@ -113,7 +117,7 @@ The canonical detailed ownership map lives in [`docs/architecture.md`](./docs/ar
|
|
|
113
117
|
- Keep direct `node:*` file-operation dependencies out of `index.ts` when an owning domain exists; the entrypoint should compose ports while domains own local filesystem details such as temp-dir preparation, attachment stats, and turn image reads
|
|
114
118
|
- In `index.ts`, prefer namespace imports for local bridge domains so orchestration reads as domain-scoped calls such as `Queue.*`, `Turns.*`, and `Rendering.*` instead of long flat import lists
|
|
115
119
|
- Keep the local `index.ts` plus `/lib/*.ts` import graph acyclic; `tests/invariants.test.ts` guards this boundary plus shared-bucket bans, empty interface-extension shell regressions, pi SDK centralization, source-only entrypoint Node-runtime/local-adapter/process/direct-pi access avoidance, runtime-domain isolation, structural leaf-domain import isolation, menu/model boundary drift, API/config default coupling, structural update/media coupling to API transport shapes, and attachment coupling to queue/inbound media/API helpers as domains keep evolving
|
|
116
|
-
- Do not reintroduce shared bucket domains such as `lib/constants.ts
|
|
120
|
+
- Do not reintroduce shared bucket domains such as `lib/constants.ts`, `lib/types.ts`, `lib/globals.ts`, or broad global-augmentation files; constants, registry keys, state interfaces, and concrete transport shapes should stay in their owning domains, and `index.ts` should not grow new shared magic constants
|
|
117
121
|
- Keep remaining `index.ts` code focused on cross-domain adapter wiring that needs live extension state, pi callbacks, Telegram API ports, or status updates; do not extract one-off closures solely to reduce line count
|
|
118
122
|
- Domain-specific queue planning, preview transport/controller behavior, rendering, Telegram API transport, menu state, and command behavior should stay in their owning domains instead of moving to `/lib/runtime.ts` solely to shrink `index.ts`
|
|
119
123
|
- Prefer narrow structural runtime ports in domains that only store or route pi-compatible values; direct pi SDK/model imports should stay centralized in `/lib/pi.ts`, while domains that actively register pi hooks/tools/commands should consume those concrete contracts through the adapter
|
|
@@ -135,10 +139,10 @@ The canonical detailed ownership map lives in [`docs/architecture.md`](./docs/ar
|
|
|
135
139
|
- For `/telegram-setup`, prefer the locally saved bot token over environment variables on repeat setup runs; env vars are the bootstrap path when no local token exists, and persisted `telegram.json` writes must remain atomic plus private because status/setup/polling paths may read it concurrently
|
|
136
140
|
- Command help plus prompt-template commands and status/model/thinking/queue controls are driven through `/start`'s Telegram inline application menu and callback queries; the Queue button shows the queued-item count, model-menu scope/pagination controls stay at the top under Main menu, the model pagination indicator opens a compact page picker, and thinking-menu text stays a compact heading because the current level is marked by button state; `/status`, `/model`, `/thinking`, and `/queue` are hidden compatibility shortcuts
|
|
137
141
|
- Shared inline-keyboard structure belongs to `keyboard`; application-control button labels, callback data, and callback behavior stay in `menu`/`menu-model`/`menu-thinking`/`menu-status`/`menu-queue` while core queue mechanics stay in `queue`
|
|
138
|
-
- Telegram `/settings` options should open nested detail submenus by default: checkbox options show a description plus Back,
|
|
142
|
+
- Telegram `/settings` options should open nested detail submenus by default: checkbox options show a description plus Back, `on`, and `off`; list options show Back plus selectable values. One-shot actions such as syncing may run directly without a submenu when there is no meaningful choice or description step.
|
|
139
143
|
- Inbound text/media may be transformed through configured `inboundHandlers` before queueing; legacy `attachmentHandlers` are deprecated compatibility aliases appended after `inboundHandlers`; outbound files must flow through `telegram_attach`
|
|
140
144
|
- Long Telegram text split recovery belongs to `text-groups`: keep it conservative, short-debounced, same chat/user/message-id contiguous, and gated by near-limit human text so normal rapid follow-ups and slash commands stay separate
|
|
141
|
-
- Inbound handlers and command-backed outbound handlers use command templates as the standard
|
|
145
|
+
- Public handler API matrix: use `registerTelegramInboundHandler(kind, handler)` for generic programmatic inbound transforms, `registerTelegramOutboundHandler(kind, handler)` for generic programmatic outbound transforms, `registerTelegramVoiceTranscriptionProvider()` for voice/audio STT providers, and `registerTelegramVoiceSynthesisProvider()` for TTS/voice-output providers. Inbound handlers and command-backed outbound handlers use command templates as the standard config contract; built-in outbound buttons use inline keyboards plus callback routing because no external command execution is needed
|
|
142
146
|
- Telegram prompt-template commands are discovered from π slash commands with `source: "prompt"`; π template names are mapped to Bot API-compatible aliases (`fix-tests` → `/fix_tests`), aliases that conflict with built-in bridge commands or hidden shortcuts are not displayed, prompt-template aliases stay out of the Telegram bot command menu, and the bridge expands template files before queueing because extension-originated `sendUserMessage()` bypasses π's interactive template expansion
|
|
143
147
|
- Unknown callback data not owned by pi-telegram prefixes (`tgbtn:`, `menu:`, `model:`, `thinking:`, `status:`, `queue:`, `section:`, `settings:`) may be forwarded as `[callback] <data>` after built-in handlers decline it; external extensions should follow `docs/callback-namespaces.md` and must not poll the same bot independently
|
|
144
148
|
- Command templates stay compact and shell-free: no `command` field, no shell execution, inline defaults are allowed as `{name=default}`, `template` may be a string or an ordered composition array, only `args`/`defaults` inherit into leaves, top-level `timeout` wraps composed sequences, stdout pipes to the next step's stdin by default, and multi-step work should use `template: [...]` rather than provider-specific fields; `pipe` is only a legacy local alias
|
package/BACKLOG.md
CHANGED
|
@@ -2,6 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
## Open Work
|
|
4
4
|
|
|
5
|
+
- [ ] Adapt `pi-xai-voice` to the finalized voice-provider contract.
|
|
6
|
+
- Priority: High after `0.11.0` lands.
|
|
7
|
+
- Idea: Update provider imports to `@llblab/pi-telegram/lib/voice.ts`, persist `voice.replyMode` to the same `telegram.json` location pi-telegram reads, return `transcriptText` only when the provider's transcript toggle is enabled, and rely on provider-owned OGG/Opus conversion.
|
|
8
|
+
- Exit: `pi-xai-voice` works against `pi-telegram@0.11.x` without direct `globalThis` access or fork-local import resolution.
|
|
9
|
+
|
|
5
10
|
- [ ] Explore always-available outbound Telegram tools for queued artifacts and controls.
|
|
6
11
|
- Priority: Low.
|
|
7
12
|
- Idea: Provide tools such as `telegram_attach_file` and `telegram_attach_button` that can be called outside an active Telegram turn, using the paired chat/session as the delivery target when safe.
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,30 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.11.0: Voice Provider Platform
|
|
4
|
+
|
|
5
|
+
- `[Voice Synthesis Provider API]` Added a first-class voice synthesis provider surface for Telegram voice replies. Providers register with `registerTelegramVoiceSynthesisProvider()` from `@llblab/pi-telegram/lib/voice.ts`, synthesize text into `.ogg`/`.opus`, may return `{ audioPath, transcriptText }`, and can contribute voice-specific prompt guidance through `getVoicePromptContribution(view)`.
|
|
6
|
+
- `[Voice Prompt Context]` Replaced the redundant `[The user sent a voice message.]` prompt marker with compact voice context owned by pi-telegram: `[voice] reply mode: manual`, `[voice] reply mode: mirror`, or `[voice] reply mode: always`. The marker is placed after handler `[outputs]` when present, otherwise after `[attachments]`, and can expand to a `[voice]` list when more fields are added.
|
|
7
|
+
- `[Voice Reply Policy]` Missing or invalid `telegram.json` `voice.replyMode` now resolves to `manual` regardless of provider defaults. Provider UIs can still change policy by writing `voice.replyMode` to the same config file pi-telegram reads.
|
|
8
|
+
- `[Voice Reply Policy]` Added `voice.replyMode` with `manual`, `mirror`, and `always` modes. The bridge tags voice turns, suppresses previews for voice-tagged replies, and transparently converts implicit assistant text to voice when policy asks for it while explicit `<!-- telegram_voice -->` markup still wins.
|
|
9
|
+
- `[Status UI]` Removed extension-section diagnostics from the Telegram status text. Section state belongs on dynamic section button labels and submenus, while `/telegram-status` keeps runtime/transport diagnostics focused.
|
|
10
|
+
- `[Extension Sections]` Main-menu section rows now support a dynamic `getLabel()` function, matching Settings rows, so extensions can surface live state directly on their button labels.
|
|
11
|
+
- `[Prompt Guidance]` Clarified `[voice]` turn context in the Telegram system prompt: `manual` means normal agent-authored output with optional explicit `telegram_voice` markup, `mirror` means voice input prefers voice output, and `always` means replies should stay TTS-friendly for automatic conversion.
|
|
12
|
+
- `[Config Interop]` Added a narrow live config runtime so companion voice sections can update `telegram.json` voice policy and the active pi-telegram config store in one step.
|
|
13
|
+
- `[Voice Delivery]` Restored outbound `type: "voice"` command handlers as the explicit first leg of voice delivery, followed by programmatic handlers and registered voice synthesis providers as zero-config fallbacks, so operator-configured `telegram.json` TTS handlers are never overridden by provider extensions.
|
|
14
|
+
- `[Inbound Handler API]` Added `registerTelegramInboundHandler(kind, handler)` as the generic programmatic counterpart to configured `inboundHandlers`, completing the handler/provider matrix beside `registerTelegramOutboundHandler`, `registerTelegramVoiceTranscriptionProvider`, and `registerTelegramVoiceSynthesisProvider`.
|
|
15
|
+
- `[Voice Transcription Provider API]` Added `registerTelegramVoiceTranscriptionProvider()` for provider-owned STT. Explicit inbound handlers and programmatic inbound handlers still run first; registered STT providers are fallback for voice/audio files without handler output.
|
|
16
|
+
- `[Settings UI]` Added built-in voice reply mode controls to pi-telegram Settings and removed the need for provider extensions to own duplicate reply-policy UI; the selector persists `voice.replyMode` to `telegram.json` even from stale visible menu messages and uses the `👄 Voice reply: hidden|manual|mirror|always` row with lowercase model-style active dots.
|
|
17
|
+
- `[Voice Prompt Context]` Missing or invalid `voice.replyMode` is now surfaced in Settings as `hidden`: it behaves like `manual`, emits no `[voice] reply mode: manual` prompt-context block, and stores no `voice.replyMode`; explicit `manual` keeps the same behavior but renders context. `mirror` mode text-originated turns stay on the manual text path, including support for explicit `telegram_voice` markup.
|
|
18
|
+
- `[Voice Delivery]` Voice delivery now uses Telegram `sendVoice` plus the native `record_voice` chat action. Providers own speech rewriting, TTS, and OGG/Opus conversion; non-OGG provider output fails voice delivery and falls back to the planned text reply.
|
|
19
|
+
- `[Voice Fallbacks]` Voice artifact failures now throw to the queue runtime, which records diagnostics and sends the planned text fallback with outbound markup stripped and reply markup preserved when no text was already delivered.
|
|
20
|
+
- `[Voice Platform Cleanup]` Collapsed merged prototype-only domains: removed shared `globals`, `global-augmentations`, and broad `shutdown` cleanup modules. Voice, section, external-handler, and outbound-handler global registry keys are now owned by their respective domains, and session shutdown no longer clears every extension registry globally.
|
|
21
|
+
- `[Docs]` Added `docs/voice.md` and README coverage for voice modes, provider registration, STT provider fallbacks, caption-style transcripts, native voice format requirements, fallback behavior, and provider-owned settings.
|
|
22
|
+
- `[Tests]` Added voice policy, provider registry, preview suppression, artifact delivery, fallback, OGG/Opus validation, prompt contribution, and entrypoint/invariant regressions. Full validation passes with 575 tests.
|
|
23
|
+
|
|
24
|
+
## 0.10.8: Compact Typing Timing Hotfix
|
|
25
|
+
|
|
26
|
+
- `[Compaction]` Telegram `/compact` now starts the native `typing` chat-action keepalive after the "Compaction started" notice is sent, then stops it on completion or failure. Impact: operators see the same Telegram activity indicator during context compression that they already see during normal agent/tool work, without showing `typing` before the explicit start confirmation arrives.
|
|
27
|
+
|
|
3
28
|
## 0.10.7: Stale Context Hardening Hotfix
|
|
4
29
|
|
|
5
30
|
- `[Session Reloads]` Context-sensitive command, pairing, queue, session-start, and update-dispatch paths now ignore only stale-session/stale-context failures instead of swallowing broad runtime errors. Impact: the bridge survives ctx replacement/fork/reload races while real bugs still surface for diagnostics.
|
|
@@ -135,7 +160,7 @@
|
|
|
135
160
|
|
|
136
161
|
## 0.9.0: Hidden Settings And Proactive Push
|
|
137
162
|
|
|
138
|
-
- `[Settings Menu]` Added hidden Telegram `/settings` with a proactive push checkbox detail submenu plus `/telegram-settings` in the terminal. Impact: operators can see green/black binary flag state, use green/black/yellow
|
|
163
|
+
- `[Settings Menu]` Added hidden Telegram `/settings` with a proactive push checkbox detail submenu plus `/telegram-settings` in the terminal. Impact: operators can see green/black binary flag state, use green/black/yellow on/off checkbox controls from Telegram, and toggle the same proactive push flag locally without adding a visible bot-command entry.
|
|
139
164
|
- `[Proactive Push]` `telegram.json` now supports `proactivePush`; when enabled, successful local non-Telegram π final replies are sent to the paired Telegram chat if no Telegram turn is active and the current session still owns the Telegram lock. Local prompt text stays private because the bot does not own or mirror terminal user messages. Impact: long local tasks can notify the phone with result context without leaking from stale bridge owners or failed/aborted turns.
|
|
140
165
|
- `[Queue UI]` Empty queue states now use the bottom-filled `⌛` hourglass while non-empty queue states keep `⏳`. Queue item details now show the selected queue position above the raw prompt preview, preserve reaction-specific priority emoji in the heading, and use side-by-side Priority/Normal tabs that refresh the heading marker immediately. The terminal status bar now stays yellow active while Telegram-owned work still has running tools even if a queued prompt is removed by reaction. Impact: queue emptiness has a small visual easter egg, item submenus stay oriented without changing queue semantics, and queue-removal reactions no longer visually degrade active work to connected.
|
|
141
166
|
- `[Model Menu]` Model rows now open a detail submenu with Back, ☑️ Activate/🟢 Active selection, and yellow/black-marked Scoped/All membership tabs. Impact: model selection remains one tap away while scoped model membership can be managed from Telegram.
|
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
**Telegram runtime adapter for π.**
|
|
6
6
|
|
|
7
|
-
`pi-telegram` turns a private Telegram DM into a session-local operator console for π. It admits work, preserves context, streams readable replies, keeps busy sessions usable through queues, lets other extensions share one bot, and turns assistant-authored intent into native Telegram artifacts.
|
|
7
|
+
`pi-telegram` turns a private Telegram DM into a session-local operator console for π. It admits work, preserves context, streams readable replies, keeps busy sessions usable through queues, lets other extensions share one bot, and turns assistant-authored intent into native Telegram artifacts. In `0.11.0`, it also becomes a voice-provider platform: companion extensions can supply Telegram transcription and synthesis providers while `pi-telegram` keeps ownership of transport, queueing, and reply policy.
|
|
8
8
|
|
|
9
9
|
This repository is an actively maintained fork of [`badlogic/pi-telegram`](https://github.com/badlogic/pi-telegram). It started from upstream commit [`cb34008`](https://github.com/badlogic/pi-telegram/commit/cb34008460b6c1ca036d92322f69d87f626be0fc) and has since diverged substantially.
|
|
10
10
|
|
|
@@ -76,7 +76,7 @@ What it feels like:
|
|
|
76
76
|
- Fire off three tasks while π is busy. They become visible queue items instead of terminal noise.
|
|
77
77
|
- Open Queue from the menu, inspect waiting work, delete stale prompts, or move important work forward.
|
|
78
78
|
- Switch models from Telegram mid-run; the adapter schedules a safe continuation instead of tearing state apart.
|
|
79
|
-
- Send a voice note;
|
|
79
|
+
- Send a voice note; a configured inbound handler or registered STT provider transcribes it; π answers in the same chat.
|
|
80
80
|
- Drop a screenshot and ask, "what is broken here?" The image payload reaches π with the local file context.
|
|
81
81
|
- Ask for a generated file; when π calls `telegram_attach`, the artifact returns to Telegram with the next reply.
|
|
82
82
|
|
|
@@ -85,7 +85,7 @@ What it feels like:
|
|
|
85
85
|
Use these inside the Telegram DM with your bot. The main entrypoint is `/start`: it opens the operator menu and exposes many of the important agent controls that normally live in the CLI, adapted for Telegram.
|
|
86
86
|
|
|
87
87
|
- **`/start`**: Pair the first Telegram user when needed, register bot commands, and open the inline application menu with command help, prompt-template commands, status rows, model controls, thinking controls, settings, and queue controls.
|
|
88
|
-
- **`/compact`**: Start session compaction when the session is idle.
|
|
88
|
+
- **`/compact`**: Start session compaction when the session is idle; Telegram shows the native typing indicator while compaction is running.
|
|
89
89
|
- **`/next`**: Dispatch the next queued turn, aborting π first if needed.
|
|
90
90
|
- **`/continue`**: Enqueue a priority `continue` prompt.
|
|
91
91
|
- **`/abort`**: Abort the active run without touching the queue.
|
|
@@ -132,11 +132,11 @@ Rendering is phone-aware: tables and lists stay narrow, table padding accounts f
|
|
|
132
132
|
|
|
133
133
|
Telegram replies to earlier text or caption messages are forwarded as `[reply]` context for normal prompts, while slash commands still parse from the new message text only. If a Telegram message is edited while still waiting in the queue, the queued turn is updated instead of duplicated. Very long text messages that Telegram appears to split automatically are coalesced through a conservative debounce when the first chunk is near Telegram's text limit.
|
|
134
134
|
|
|
135
|
-
### Inbound handlers
|
|
135
|
+
### Inbound handlers and STT providers
|
|
136
136
|
|
|
137
137
|
`telegram.json` can define ordered `inboundHandlers` for Telegram → π preprocessing: text translation, voice transcription, OCR, PDF extraction, or any command-template pipeline. Matching handlers run before the turn enters the queue; failed handlers record diagnostics and fall back safely. Legacy `attachmentHandlers` still work as a deprecated compatibility alias appended after `inboundHandlers`.
|
|
138
138
|
|
|
139
|
-
A practical voice setup is simple: Telegram `.ogg` arrives, STT runs locally or through your chosen command, stdout is injected as `[outputs]`, and π receives the result as usable prompt context.
|
|
139
|
+
A practical voice setup is simple: Telegram `.ogg` arrives, STT runs locally or through your chosen command, stdout is injected as `[outputs]`, and π receives the result as usable prompt context. Extensions can also register programmatic inbound handlers; full voice extensions can register transcription providers. Explicit `inboundHandlers` and legacy `attachmentHandlers` run first, then programmatic inbound handlers, then registered STT providers as fallback for voice/audio files.
|
|
140
140
|
|
|
141
141
|
```json
|
|
142
142
|
{
|
|
@@ -163,7 +163,7 @@ A practical voice setup is simple: Telegram `.ogg` arrives, STT runs locally or
|
|
|
163
163
|
}
|
|
164
164
|
```
|
|
165
165
|
|
|
166
|
-
### Outbound handlers, voice, and buttons
|
|
166
|
+
### Outbound handlers, voice synthesis providers, and buttons
|
|
167
167
|
|
|
168
168
|
Assistant replies can include hidden outbound blocks. `telegram_voice` and `telegram_button` are not π tools; they are assistant-authored HTML comments that the adapter removes from Telegram text and handles after `agent_end`. Recognized blocks must start at column zero on a top-level line outside fenced code, quotes, and lists.
|
|
169
169
|
|
|
@@ -179,27 +179,50 @@ List the main risks first.
|
|
|
179
179
|
-->
|
|
180
180
|
```
|
|
181
181
|
|
|
182
|
-
Outbound `type: "text"` handlers can transform final text/Markdown before Telegram rendering and delivery.
|
|
182
|
+
Outbound `type: "text"` handlers can transform final text/Markdown before Telegram rendering and delivery. Voice output can be handled either by configured `outboundHandlers` with `type: "voice"` or by registered voice synthesis provider extensions: the bridge extracts `telegram_voice` text or intercepts text by reply mode, asks the voice pipeline for a `.ogg`/`.opus` artifact, and uploads it through Telegram `sendVoice`. Explicit configured voice handlers run before zero-config providers, so operator-owned `telegram.json` pipelines stay authoritative.
|
|
183
183
|
|
|
184
|
-
|
|
184
|
+
The agent writes intent; providers or voice handlers own TTS and format conversion, the adapter owns Telegram transport, and buttons route back as queued prompts.
|
|
185
185
|
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
186
|
+
### Voice reply policies
|
|
187
|
+
|
|
188
|
+
The bridge can automatically convert agent text replies into Telegram voice messages without requiring explicit `<!-- telegram_voice -->` markup in every response. Configure this from Settings → `👄 Voice reply` or by setting `voice.replyMode` in `telegram.json`:
|
|
189
|
+
|
|
190
|
+
- `hidden` (default): no `voice.replyMode` is stored. Behavior is manual, but prompt context stays silent.
|
|
191
|
+
- `manual`: agent-authored `<!-- telegram_voice -->` markup is required for voice replies; no automatic conversion. Unlike `hidden`, this explicit mode adds `[voice] reply mode: manual` context.
|
|
192
|
+
- `mirror`: when the user sends a voice message, the next reply is converted to voice and text preview is suppressed. Text-originated turns stay on the normal/manual text path, so agent-authored `<!-- telegram_voice -->` markup still works explicitly.
|
|
193
|
+
- `always`: every reply is converted to voice and text preview is suppressed.
|
|
194
|
+
|
|
195
|
+
If `telegram.json` explicitly sets a valid `voice.replyMode`, prompts include compact `[voice] reply mode: ...` context after handler outputs. When the field is missing or invalid, behavior still defaults to manual/hidden and the prompt context stays silent.
|
|
196
|
+
|
|
197
|
+
In `mirror` and `always` modes, the bridge transparently intercepts agent text responses and routes them through the outbound voice pipeline. Configured `outboundHandlers` with `type: "voice"` run first in their configured order; zero-config registered synthesis providers run after them as progressive fallbacks. If several synthesis providers are installed, they are tried in registration order and the first one that returns a valid `.ogg`/`.opus` artifact handles the reply; `undefined`, errors, or invalid output fall through to the next provider. If every voice generator fails, the bridge falls back to sending the text reply instead.
|
|
198
|
+
|
|
199
|
+
Voice synthesis provider extensions (e.g. `pi-xai-voice`) register a TTS backend at runtime:
|
|
200
|
+
|
|
201
|
+
```typescript
|
|
202
|
+
import { registerTelegramVoiceSynthesisProvider } from "@llblab/pi-telegram/lib/voice.ts";
|
|
203
|
+
import { recordTelegramRuntimeEvent } from "@llblab/pi-telegram/lib/outbound-handlers.ts";
|
|
204
|
+
|
|
205
|
+
// Return path only (backward compatible)
|
|
206
|
+
const dispose = registerTelegramVoiceSynthesisProvider(async (text, { lang, rate }) => {
|
|
207
|
+
const path = await myTTS(text, { language: lang });
|
|
208
|
+
return path; // must be .ogg or .opus
|
|
209
|
+
});
|
|
210
|
+
|
|
211
|
+
// Return path + transcript caption
|
|
212
|
+
const dispose2 = registerTelegramVoiceSynthesisProvider(async (text, { lang, rate }) => {
|
|
213
|
+
const rewritten = rewriteWithSpeechTags(text); // internal TTS optimization
|
|
214
|
+
const path = await myTTS(rewritten, { language: lang });
|
|
215
|
+
return { audioPath: path, transcriptText: text };
|
|
216
|
+
});
|
|
217
|
+
|
|
218
|
+
// Surface diagnostics in /telegram-status
|
|
219
|
+
recordTelegramRuntimeEvent("xai-voice", new Error("TTS complete"), {
|
|
220
|
+
phase: "tts",
|
|
221
|
+
durationMs: 1200,
|
|
222
|
+
});
|
|
200
223
|
```
|
|
201
224
|
|
|
202
|
-
|
|
225
|
+
Multiple synthesis providers can be registered; the bridge tries configured `type: "voice"` handlers first, then programmatic handlers, then registered synthesis providers in registration order until one succeeds. Providers and handlers receive the text to synthesize and optional `lang`/`rate` hints from `<!-- telegram_voice -->` markup or the automatic interception path. Voice delivery must produce `.ogg` or `.opus` files.
|
|
203
226
|
|
|
204
227
|
### Extension interop
|
|
205
228
|
|
|
@@ -209,7 +232,7 @@ Unknown inline-button callbacks are forwarded to π as `[callback] <data>` when
|
|
|
209
232
|
|
|
210
233
|
Ordinary pi extensions can register structured UI sections that appear in the main Telegram menu and Settings submenu without owning a second poller. Each section gets a narrow typed context with `edit`, `open`, `enqueuePrompt`, `answerCallback`, and `callbackData()` — enough to build interactive Telegram-native surfaces while `pi-telegram` owns transport, callback routing, navigation hierarchy, and diagnostics.
|
|
211
234
|
|
|
212
|
-
Import from `@llblab/pi-telegram
|
|
235
|
+
Import `registerTelegramSection()` from `@llblab/pi-telegram/lib/extension-sections.ts` and return a disposer on shutdown. Sections can send interactive messages directly into the chat via `ctx.open()` — confirmation dialogs, approve/deny gates, and multi-step forms live outside the menu hierarchy while callbacks route through the same typed handler. See [`@llblab/pi-telegram-extension-demo`](https://github.com/llblab/pi-telegram-extension-demo) for a working reference and the [Extension Sections Standard](./docs/extension-sections.md) for the full contract.
|
|
213
236
|
|
|
214
237
|
### Proactive push
|
|
215
238
|
|
|
@@ -224,6 +247,7 @@ Import from `@llblab/pi-telegram`, call `registerTelegramSection()`, and return
|
|
|
224
247
|
- [Architecture](./docs/architecture.md): runtime and subsystem overview.
|
|
225
248
|
- [Inbound Handlers](./docs/inbound-handlers.md): Telegram → π preprocessing.
|
|
226
249
|
- [Outbound Handlers](./docs/outbound-handlers.md): final text, voice, and artifact pipelines.
|
|
250
|
+
- [Voice Integration](./docs/voice.md): voice reply policies, transparent interception, and provider extension API.
|
|
227
251
|
- [Command Templates](./docs/command-templates.md): portable command-template contract.
|
|
228
252
|
- [Callback Namespaces](./docs/callback-namespaces.md): callback interop for layered extensions.
|
|
229
253
|
- [External Handlers](./docs/external-handlers.md): shared update interception.
|
package/docs/README.md
CHANGED
|
@@ -6,8 +6,9 @@ Living index of project documentation in `/docs`.
|
|
|
6
6
|
|
|
7
7
|
- [architecture.md](./architecture.md) — Overview of the Telegram bridge runtime, queueing model, rendering pipeline, and interactive controls
|
|
8
8
|
- [command-templates.md](./command-templates.md) — Portable command-template standard core
|
|
9
|
-
- [inbound-handlers.md](./inbound-handlers.md) — Local `pi-telegram` inbound text/media handler bus, legacy `attachmentHandlers` compatibility, placeholders, and fallbacks
|
|
10
|
-
- [outbound-handlers.md](./outbound-handlers.md) — Local `pi-telegram` outbound-handler config, text/voice/button behavior, artifact outputs, and callback routing
|
|
9
|
+
- [inbound-handlers.md](./inbound-handlers.md) — Local `pi-telegram` inbound text/media handler bus, programmatic inbound handlers, registered STT provider fallbacks, legacy `attachmentHandlers` compatibility, placeholders, and fallbacks
|
|
10
|
+
- [outbound-handlers.md](./outbound-handlers.md) — Local `pi-telegram` outbound-handler config, text/voice/button behavior, voice synthesis provider fallback priority, artifact outputs, and callback routing
|
|
11
|
+
- [voice.md](./voice.md) — Voice integration guide: detection, reply policy, STT/TTS provider registration, provider-owned conversion, and transparent interception
|
|
11
12
|
- [locks.md](./locks.md) — Shared `locks.json` standard for singleton extension ownership
|
|
12
13
|
- [callback-namespaces.md](./callback-namespaces.md) — Shared Telegram `callback_data` namespace standard for layered extensions
|
|
13
14
|
- [external-handlers.md](./external-handlers.md) — Runtime interceptor registry that lets layered extensions observe and consume Telegram updates without owning their own polling connection
|
package/docs/architecture.md
CHANGED
|
@@ -28,7 +28,7 @@ Current runtime areas use these ownership boundaries:
|
|
|
28
28
|
- `config` / `setup`: persisted bot/session pairing state, authorization, first-user pairing, token prompting, env fallback, validation, and config persistence.
|
|
29
29
|
- `locks` / `polling`: singleton `locks.json` ownership, takeover/restart semantics, long-poll controller state, update offset persistence, and poll-loop runtime wiring.
|
|
30
30
|
- `updates` / `routing`: update classification/execution planning, paired authorization, reactions, edits, callbacks, and inbound route composition.
|
|
31
|
-
- `media` / `text-groups` / `turns` / `inbound-handlers`: text/media extraction, media-group debounce, long-text split coalescing, inbound downloads, inbound text/media handler execution, turn building/editing, image reads, and legacy `attachmentHandlers` compatibility.
|
|
31
|
+
- `media` / `text-groups` / `turns` / `inbound-handlers`: text/media extraction, media-group debounce, long-text split coalescing, inbound downloads, configured and programmatic inbound text/media handler execution, turn building/editing, image reads, and legacy `attachmentHandlers` compatibility.
|
|
32
32
|
- `queue`: queue item contracts, lane admission/order, stores, mutations, dispatch readiness/runtime, prompt/control enqueueing, and session/agent/tool lifecycle sequencing.
|
|
33
33
|
- `runtime`: session-local coordination primitives: counters, lifecycle flags, setup guard, abort handler, typing-loop timers, prompt-dispatch flags, and agent-end reset binding.
|
|
34
34
|
- `model` / `menu-model` / `menu-thinking` / `menu-status` / `menu` / `menu-queue` / `menu-settings` / `commands`: model identity/thinking levels, scoped model resolution, in-flight switching, model/thinking/status/queue/settings menu UI, inline application callback composition, slash commands, and bot command registration.
|
|
@@ -114,7 +114,7 @@ Dispatch is gated by:
|
|
|
114
114
|
|
|
115
115
|
This prevents queue races around rapid follow-ups, `/compact`, and mixed local plus Telegram activity. Post-agent-end dispatch retries are scheduled through a session-bound deferred dispatcher that activates on session start, cancels timers on session shutdown, and skips callbacks from older generations before they touch `ExtensionContext`. Telegram `/start` and hidden compatibility shortcuts `/status`, `/model`, `/thinking`, `/queue`, and `/settings` execute immediately; the dispatch controller still serializes any deferred control items so a queued control action must settle before the next queued action can dispatch.
|
|
116
116
|
|
|
117
|
-
`/start` opens the main application menu: visible command help, compact command-only prompt-template rows when π exposes Telegram-compatible prompt-template names, status rows (`Status`, `Usage`, `Cost`, `Context`), and top-level buttons for model, thinking, and queue sections. The `Status` row reports `compacting` while a Telegram `/compact` run is active. The Queue button includes the current queued-item count. Hidden compatibility shortcuts `/help`, `/status`, `/model`, `/thinking`, and `/queue` jump directly to their corresponding menu screens, while `/settings` opens the hidden settings menu for bridge toggles such as proactive push. Settings options open detail submenus; checkbox-like settings use Back plus green/black/yellow
|
|
117
|
+
`/start` opens the main application menu: visible command help, compact command-only prompt-template rows when π exposes Telegram-compatible prompt-template names, status rows (`Status`, `Usage`, `Cost`, `Context`), and top-level buttons for model, thinking, and queue sections. The `Status` row reports `compacting` while a Telegram `/compact` run is active, and the bridge sends Telegram's native `typing` chat action as a keepalive for the same compaction window. The Queue button includes the current queued-item count. Hidden compatibility shortcuts `/help`, `/status`, `/model`, `/thinking`, and `/queue` jump directly to their corresponding menu screens, while `/settings` opens the hidden settings menu for bridge toggles such as proactive push. Settings options open detail submenus; checkbox-like settings use Back plus green/black/yellow `on` and `off` controls instead of mutating directly from the list. Command emoji come from the `commands` domain map so visible command descriptions and matching menu buttons share one fixed adornment source. Prompt-template commands use a fixed `🧩` marker, map π template names to Telegram-safe aliases such as `fix-tests` → `/fix_tests`, stay visible only inside the `/start` menu, and expand before queueing because `ExtensionAPI.sendUserMessage()` intentionally bypasses π prompt-template expansion for extension-originated messages. Every submenu starts with a top Back row so navigation stays anchored near the original user message above the inline keyboard; model-menu pagination controls sit near the top, tapping the pagination indicator opens a compact page picker headed by `<b>Choose a page:</b>`, and tapping a model opens a detail submenu with Back, ☑️ Activate/🟢 Active selection, and yellow/black-marked Scoped/All membership tabs. `menu-model` owns model-menu state, scoped model pages, model detail rendering, scoped-list persistence planning, and model-menu rendering while `model` owns core model identity/switching semantics. `menu-thinking` owns thinking-menu text, reply markup, callback handling, and message rendering. `menu-status` owns status-menu payloads, status callback handling, and status-message rendering. `menu-queue` owns queue-menu UI only: queue items are rendered under a compact `<b>Queue:</b>` heading, top-to-bottom in dispatch order, numbered, and marked with `⚡` for priority prompts or `📎` for prompts with attachments. An empty queue renders bold message text with the bottom-filled `⌛` hourglass plus the top Main menu button, while non-empty queue states keep the running `⏳` hourglass. Selecting an item opens a submenu that displays the queue item number above the full queued prompt text with Back, side-by-side Priority/Normal tabs, and Cancel. If a callback targets an item that has already left the queue, the menu refreshes the list instead of applying a stale mutation.
|
|
118
118
|
|
|
119
119
|
### Abort Behavior
|
|
120
120
|
|
|
@@ -157,7 +157,7 @@ Telegram prompt responses use explicit delivery context to attach outbound text,
|
|
|
157
157
|
|
|
158
158
|
Outbound files are sent only after the active Telegram turn completes, must be staged through the `telegram_attach` tool, are staged atomically per tool call, are checked against a default 50 MiB limit configurable through `PI_TELEGRAM_OUTBOUND_ATTACHMENT_MAX_BYTES` or `TELEGRAM_MAX_ATTACHMENT_SIZE_BYTES`, and use file-backed multipart blobs so large sends do not require preloading whole files into memory.
|
|
159
159
|
|
|
160
|
-
Assistant-authored outbound actions use final-message markup instead of agent tool calls. Preview updates strip closed top-level HTML comments and currently open/partial top-level comment starts before rendering, so users do not see transient metadata even when streaming flushes happen after only `<`, `<!`, or `<!--`. On `agent_end`, the bridge removes top-level comments from the Markdown text reply, but treats column-zero top-level `<!-- telegram_voice ... -->` and `<!-- telegram_button ... -->` blocks specially before delivery; comments inside fenced code, quotes, lists, or indented examples stay literal, including fenced blocks with Markdown-valid indented closing fences. Voice
|
|
160
|
+
Assistant-authored outbound actions use final-message markup instead of agent tool calls. Preview updates strip closed top-level HTML comments and currently open/partial top-level comment starts before rendering, so users do not see transient metadata even when streaming flushes happen after only `<`, `<!`, or `<!--`. On `agent_end`, the bridge removes top-level comments from the Markdown text reply, but treats column-zero top-level `<!-- telegram_voice ... -->` and `<!-- telegram_button ... -->` blocks specially before delivery; comments inside fenced code, quotes, lists, or indented examples stay literal, including fenced blocks with Markdown-valid indented closing fences. Voice uses a single fallback pipeline: configured `outboundHandlers` with `type: "voice"`, then programmatic `voice` handlers, then registered synthesis providers from `lib/voice.ts`. The bridge extracts body text, `text="..."`, or colon shorthand, asks the pipeline for an `.ogg`/`.opus` artifact, validates native voice format, and uploads the generated file via Telegram `sendVoice`; when delivery fails, the queue runtime records diagnostics and falls back to the planned text reply when no text was already delivered. Synthesis providers own TTS, speech rewriting, transcript choice, and format conversion. Button blocks are built in: each `telegram_button` block becomes one inline-keyboard button on the final text, and callback clicks enqueue the configured prompt text as a normal Telegram prompt turn; the `telegram_button: Label` shorthand uses the same text for label and prompt, `prompt="..."` supports explicit one-line prompts, and body-form buttons use the body as the prompt. Unknown callback data that does not match pi-telegram-owned prefixes (`tgbtn:`, `menu:`, `model:`, `thinking:`, `status:`, `queue:`, future `section:`) is forwarded to π as `[callback] <data>` after built-in handlers decline it, giving layered extensions a simple namespaced button channel without separate polling; layered callback payloads should follow the [Callback Namespace Standard](./callback-namespaces.md). Future structured menu integrations should use the [Telegram Extension Sections Standard](./extension-sections.md) instead of hand-rolled fallback callbacks. When proactive push is enabled, successful local non-Telegram final replies are sent to the paired chat. Local prompt text is not sent because the bot does not own or mirror terminal user messages. This keeps terminal-originated results visible in Telegram without changing Telegram-originated turn delivery.
|
|
161
161
|
|
|
162
162
|
This keeps technical Markdown, code, tables, formulas, and numbered lists in the text channel when appropriate while allowing TTS-friendly voice messages and tappable continuations without invoking `telegram_attach` or extra transport tools. Telegram prompt guidance targets about 37 visible cells for tables, dense list items, and compact text blocks because emoji and other wide glyphs make raw character counts misleading on mobile screens.
|
|
163
163
|
|
|
@@ -47,6 +47,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
47
47
|
id: "@llblab/pi-telegram-extension-demo",
|
|
48
48
|
label: "🧪 Demo submenu",
|
|
49
49
|
order: 10,
|
|
50
|
+
getLabel: () => `${flag ? "🟢" : "⚫️"} Demo submenu`,
|
|
50
51
|
render: async (ctx) => ({
|
|
51
52
|
text: "<b>Demo</b>",
|
|
52
53
|
parseMode: "html",
|
|
@@ -88,6 +89,7 @@ interface TelegramSectionRegistration {
|
|
|
88
89
|
id: TelegramSectionId;
|
|
89
90
|
label: string;
|
|
90
91
|
order?: number;
|
|
92
|
+
getLabel?: () => string;
|
|
91
93
|
render: (
|
|
92
94
|
ctx: TelegramSectionContext,
|
|
93
95
|
) => TelegramSectionView | Promise<TelegramSectionView>;
|
|
@@ -137,7 +139,7 @@ import { registerTelegramSection } from "../pi-telegram/lib/extension-sections.t
|
|
|
137
139
|
|
|
138
140
|
**Load order:** `pi-telegram` must load first (sets the global registry). Demo/consumer extensions load second (call `registerTelegramSection`). Pi's normal extension loader guarantees this when `pi-telegram` is listed first.
|
|
139
141
|
|
|
140
|
-
**Shutdown:** Call `pi.on("shutdown", () => unregister())` to clean up.
|
|
142
|
+
**Shutdown:** Call `pi.on("shutdown", () => unregister())` to clean up your section. `pi-telegram` owns the registry for its loaded session, but it does not globally wipe extension registries on every `session_shutdown`.
|
|
141
143
|
|
|
142
144
|
## 6. Menu Integration
|
|
143
145
|
|
|
@@ -145,13 +147,13 @@ Sections appear in two locations:
|
|
|
145
147
|
|
|
146
148
|
### Main menu
|
|
147
149
|
|
|
148
|
-
Section rows are injected **before the ⚙️ Settings row**. Ordered by `order` (lower first), then `id` alphabetically.
|
|
150
|
+
Section rows are injected **before the ⚙️ Settings row**. Ordered by `order` (lower first), then `id` alphabetically. The top-level `getLabel()` function (if present) is called on every render to produce a dynamic main-menu label — use it for extension status indicators.
|
|
149
151
|
|
|
150
152
|
```
|
|
151
153
|
🤖 Model: anthropic/claude-sonnet-4-5
|
|
152
154
|
🧠 Thinking: off
|
|
153
155
|
⌛ Queue: 0
|
|
154
|
-
|
|
156
|
+
🟢 Demo submenu ← extension section (dynamic label)
|
|
155
157
|
⚙️ Settings
|
|
156
158
|
```
|
|
157
159
|
|
|
@@ -396,7 +398,7 @@ interface TelegramSectionDiagnostic {
|
|
|
396
398
|
}
|
|
397
399
|
```
|
|
398
400
|
|
|
399
|
-
Available
|
|
401
|
+
Available programmatically via `getTelegramSectionDiagnostics()`. Section runtime state is not shown in Telegram status text; sections should surface user-facing state through dynamic button labels and their own submenus.
|
|
400
402
|
|
|
401
403
|
## 13. Purpose and Non-Goals
|
|
402
404
|
|
package/docs/inbound-handlers.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Inbound Handlers
|
|
2
2
|
|
|
3
|
-
`pi-telegram` can run ordered inbound handlers before a Telegram turn enters the π queue. Inbound handlers are the provider-neutral Telegram → π transformation bus for raw text and downloaded media/files.
|
|
3
|
+
`pi-telegram` can run ordered inbound handlers before a Telegram turn enters the π queue. Inbound handlers are the provider-neutral Telegram → π transformation bus for raw text and downloaded media/files. Extensions can also register programmatic inbound handlers with `registerTelegramInboundHandler()`, and voice extensions can register STT providers as a zero-config fallback for Telegram voice/audio files.
|
|
4
4
|
|
|
5
5
|
This document is the local inbound adaptation of the portable [Command Template Standard](./command-templates.md). It is also the canonical home for the legacy `attachmentHandlers` compatibility config.
|
|
6
6
|
|
|
@@ -86,6 +86,46 @@ A handler list is ordered. For each downloaded file, matching media/file handler
|
|
|
86
86
|
|
|
87
87
|
If a matching handler fails with a non-zero exit code, the runtime records diagnostics and tries the next matching handler. If every matching handler fails, the attachment remains visible in the prompt as a normal local file reference.
|
|
88
88
|
|
|
89
|
+
## Programmatic Inbound Handlers And STT Fallbacks
|
|
90
|
+
|
|
91
|
+
Extensions can register programmatic inbound handlers with `registerTelegramInboundHandler(kind, handler)` from `@llblab/pi-telegram/lib/inbound-handlers.ts`. This is the code-level counterpart to configured `inboundHandlers`; use it for extension-owned transformations that are not voice-specific.
|
|
92
|
+
|
|
93
|
+
Voice extensions can register STT providers with `registerTelegramVoiceTranscriptionProvider()` from `@llblab/pi-telegram/lib/voice.ts`. This is the zero-config extension path for voice/audio input: an extension such as `pi-xai-voice` can transcribe Telegram voice notes without requiring the operator to write an `inboundHandlers` command template.
|
|
94
|
+
|
|
95
|
+
Priority stays explicit and predictable:
|
|
96
|
+
|
|
97
|
+
1. configured `inboundHandlers`
|
|
98
|
+
2. legacy `attachmentHandlers`
|
|
99
|
+
3. programmatic `registerTelegramInboundHandler(kind, ...)` handlers
|
|
100
|
+
4. registered STT providers for `voice`/`audio` files that still have no handler output
|
|
101
|
+
5. built-in text-file fallback for text attachments
|
|
102
|
+
|
|
103
|
+
```ts
|
|
104
|
+
import { registerTelegramInboundHandler } from "@llblab/pi-telegram/lib/inbound-handlers.ts";
|
|
105
|
+
import { registerTelegramVoiceTranscriptionProvider } from "@llblab/pi-telegram/lib/voice.ts";
|
|
106
|
+
|
|
107
|
+
const disposeInbound = registerTelegramInboundHandler("document", async ({ file }) => {
|
|
108
|
+
if (!file?.mimeType?.includes("pdf")) return undefined;
|
|
109
|
+
const text = await extractPdf(file.path);
|
|
110
|
+
return text || undefined;
|
|
111
|
+
});
|
|
112
|
+
|
|
113
|
+
const dispose = registerTelegramVoiceTranscriptionProvider(
|
|
114
|
+
async (file) => {
|
|
115
|
+
if (file.kind !== "voice" && file.kind !== "audio") return undefined;
|
|
116
|
+
const result = await transcribe(file.path);
|
|
117
|
+
return result.text
|
|
118
|
+
? { text: result.text, language: result.language }
|
|
119
|
+
: undefined;
|
|
120
|
+
},
|
|
121
|
+
{ id: "my-stt" },
|
|
122
|
+
);
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
A provider can return a plain transcript string, `{ text, language? }`, or `undefined` to pass. Provider output is injected into `[outputs]` exactly like command-template handler output. Programmatic inbound handlers and STT providers are fallbacks only; they do not override operator-configured inbound handlers.
|
|
126
|
+
|
|
127
|
+
If several programmatic inbound handlers are registered for a kind, they are tried in registration order; the first non-empty output wins for media files, while text handlers transform text sequentially. If several STT providers are registered, they are tried in registration order. The first provider that returns non-empty text wins. Providers that return `undefined` pass; providers that throw are recorded and the next provider is tried. If none produces text, the voice/audio file remains as a normal attachment reference.
|
|
128
|
+
|
|
89
129
|
## Prompt Output
|
|
90
130
|
|
|
91
131
|
Local attachments stay in the prompt under `[attachments] <directory>` with relative file entries. Successful media/file handler stdout is added under `[outputs]`. For composed media/file handlers, each step receives the previous step's stdout on stdin by default, and stdout from the last successful step is used as the handler output. Empty output and failed handler output are omitted from the prompt text.
|
|
@@ -2,21 +2,23 @@
|
|
|
2
2
|
|
|
3
3
|
`pi-telegram` maps hidden assistant-authored HTML comments to Telegram-native outbound actions.
|
|
4
4
|
|
|
5
|
-
This is intentionally prompt-driven: the agent writes normal Markdown plus small hidden top-level blocks, and the bridge performs the transport work after `agent_end`. `telegram_voice` and `telegram_button` are not π tools. Outbound behavior is an emergent result of the assistant prompt,
|
|
5
|
+
This is intentionally prompt-driven: the agent writes normal Markdown plus small hidden top-level blocks, and the bridge performs the transport work after `agent_end`. `telegram_voice` and `telegram_button` are not π tools. Outbound behavior is an emergent result of the assistant prompt, text command-template handlers, registered voice synthesis providers, generated artifacts, and reply delivery. That avoids extra agent-side tool calls, avoids fragile parameter plumbing inside the conversation, and minimizes latency because text, voice, and buttons are planned in one standard assistant reply.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Text handlers use the portable [Command Template Standard](./command-templates.md). Programmatic outbound handlers use `registerTelegramOutboundHandler(kind, handler)`. Voice replies can use configured command-template handlers or the provider API described in [Voice Integration](./voice.md).
|
|
8
8
|
|
|
9
9
|
## Standard
|
|
10
10
|
|
|
11
11
|
An outbound handler is selected by `type`. Text replies and assistant markup map to handler types:
|
|
12
12
|
|
|
13
|
-
| Source
|
|
14
|
-
|
|
|
15
|
-
| Final text
|
|
16
|
-
| `telegram_voice`
|
|
17
|
-
| `telegram_button` | Built-in
|
|
13
|
+
| Source | Handler | Action |
|
|
14
|
+
| ----------------- | ----------------------------- | ----------------------- |
|
|
15
|
+
| Final text | `outboundHandlers[type=text]` | Transform before render |
|
|
16
|
+
| `telegram_voice` | Voice pipeline | OGG/Opus `sendVoice` |
|
|
17
|
+
| `telegram_button` | Built-in | Attach inline button |
|
|
18
18
|
|
|
19
|
-
|
|
19
|
+
The voice pipeline is detailed below: configured `type: "voice"` handlers first, then programmatic handlers, then registered synthesis providers.
|
|
20
|
+
|
|
21
|
+
Configured text handlers provide `template`. A string is one command; an array is ordered composition. Top-level `args` and `defaults` apply to all composed steps unless a step defines private values. The command-template default timeout applies automatically. Legacy configs may still use `pipe`, but `template: [...]` is the preferred standard shape.
|
|
20
22
|
|
|
21
23
|
## Text Handler Config
|
|
22
24
|
|
|
@@ -52,27 +54,30 @@ Stdin-based or subagent-backed translation can omit `{text}` from the template b
|
|
|
52
54
|
|
|
53
55
|
A text handler should preserve the full message unless shortening is intentional; for translation prompts, explicitly ask the tool to keep Markdown, line breaks, and details unchanged.
|
|
54
56
|
|
|
55
|
-
## Voice
|
|
57
|
+
## Voice Delivery Priority
|
|
56
58
|
|
|
57
|
-
|
|
59
|
+
Voice replies use one fallback pipeline:
|
|
58
60
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
61
|
+
1. configured `outboundHandlers` with `type: "voice"` in `telegram.json` order
|
|
62
|
+
2. programmatic `registerTelegramOutboundHandler("voice", ...)` handlers
|
|
63
|
+
3. registered voice synthesis providers from `@llblab/pi-telegram/lib/voice.ts`
|
|
64
|
+
|
|
65
|
+
This makes provider extensions a zero-config convenience without overriding explicit operator-owned `telegram.json` handlers. If several synthesis providers are registered, they are tried in registration order; the first provider that returns a valid `.ogg`/`.opus` artifact handles the reply. Returning `undefined` passes to the next provider, while thrown errors or invalid files are recorded before the next fallback is tried.
|
|
66
|
+
|
|
67
|
+
## Voice Synthesis Provider API
|
|
68
|
+
|
|
69
|
+
Voice replies can be delivered by synthesis providers registered through `@llblab/pi-telegram/lib/voice.ts`:
|
|
70
|
+
|
|
71
|
+
```ts
|
|
72
|
+
import { registerTelegramVoiceSynthesisProvider } from "@llblab/pi-telegram/lib/voice.ts";
|
|
73
|
+
|
|
74
|
+
const dispose = registerTelegramVoiceSynthesisProvider(async (text, options) => {
|
|
75
|
+
const audioPath = await synthesizeToOggOpus(text, options);
|
|
76
|
+
return { audioPath, transcriptText: text };
|
|
77
|
+
});
|
|
73
78
|
```
|
|
74
79
|
|
|
75
|
-
|
|
80
|
+
Synthesis providers receive the extracted `telegram_voice` text plus optional `lang`/`rate` hints. They own translation, TTS, speech rewriting, transcript choice, and OGG/Opus conversion. The bridge validates that the returned file ends in `.ogg` or `.opus`, sends it through Telegram `sendVoice`, and falls back to planned text if delivery fails before any visible text was delivered. Providers run after configured and programmatic voice handlers in the priority chain above.
|
|
76
81
|
|
|
77
82
|
## Voice Markup
|
|
78
83
|
|
|
@@ -90,29 +95,7 @@ Text to synthesize as a Telegram voice message.
|
|
|
90
95
|
<!-- telegram_voice: Short spoken companion summary. -->
|
|
91
96
|
```
|
|
92
97
|
|
|
93
|
-
The bridge strips the comment from Telegram text. On `agent_end`, it maps each `telegram_voice` block to
|
|
94
|
-
|
|
95
|
-
## Built-In Voice Placeholders
|
|
96
|
-
|
|
97
|
-
Voice outbound handlers receive these runtime placeholders:
|
|
98
|
-
|
|
99
|
-
| Placeholder | Value |
|
|
100
|
-
| --- | --- |
|
|
101
|
-
| `{text}` | Voice text from body, attr, or colon form |
|
|
102
|
-
| `{lang}` | Optional override, e.g. `lang=ru` |
|
|
103
|
-
| `{rate}` | Optional override, e.g. `rate=+30%` |
|
|
104
|
-
| `{mp3}` | Temp MP3 path under agent temp |
|
|
105
|
-
| `{ogg}` | Temp OGG path under agent temp |
|
|
106
|
-
|
|
107
|
-
Temp artifacts use unique flat names such as `<uuid>-voice.mp3` and `<uuid>-voice.ogg`. The bridge does not create per-handler directory trees.
|
|
108
|
-
|
|
109
|
-
## Output
|
|
110
|
-
|
|
111
|
-
For composed handlers, `output` selects the primary artifact after the composition completes. Omitted `output` means `"stdout"`, so the final step should print the generated OGG/Opus path. `"output": "ogg"` means the generated file path comes from `{ogg}`. A value such as `"{ogg}"` is equivalent. Composition also follows the command-template standard where each step's stdout is provided as stdin to the next step by default.
|
|
112
|
-
|
|
113
|
-
For one-step `template` handlers, stdout remains the default result channel: the command should print the generated OGG/Opus path.
|
|
114
|
-
|
|
115
|
-
**Critical steps:** voice synthesis is often a multi-step transform → TTS → conversion pipeline. The final audio conversion step is inherently critical — if it fails, the voice output is invalid. Mark conversion steps as `"critical": true` when a composed handler must abort after conversion failure instead of continuing to later non-critical steps. Use multiple matching `type: "voice"` handlers when you need provider or command fallbacks. See [Command Template Standard](./command-templates.md) for semantics.
|
|
98
|
+
The bridge strips the comment from Telegram text. On `agent_end`, it maps each `telegram_voice` block to a provider call, generates one file per block, and sends each file as an independent Telegram-native voice message. The opening `<!-- telegram_voice` marker must start at column zero on a top-level line outside fenced code, quotes, and lists; otherwise it is rendered as literal Markdown. Body-form comments leave the opening line unclosed until the body-ending `-->`; closed heads can use `text="..."` for explicit one-line spoken text.
|
|
116
99
|
|
|
117
100
|
## Buttons Markup
|
|
118
101
|
|
|
@@ -149,6 +132,6 @@ The extension injects Telegram-specific system prompt guidance so agents know th
|
|
|
149
132
|
- Write the full technical answer as normal Markdown.
|
|
150
133
|
- Add `telegram_voice` when a Telegram-native voice message is useful; use body text, `text="..."`, or colon shorthand for the text to synthesize. A companion summary is optional, no specific summary format is required.
|
|
151
134
|
- Add `telegram_button: ...` when label equals prompt, `telegram_button label="..." prompt="..."` for one-line prompts, or `telegram_button label="..."` with a body for multiline prompts. If the reply contains only button/voice comment blocks, add a short visible marker (for example `Choose one:`) before them so Telegram always has a visible parent message for attachment.
|
|
152
|
-
- Do not call
|
|
135
|
+
- Do not call Telegram transport tools for voice or buttons; the bridge owns delivery, while registered voice synthesis providers own TTS and OGG/Opus conversion.
|
|
153
136
|
|
|
154
137
|
This keeps the agent focused on semantics and lets the bridge handle low-latency Telegram adaptation.
|