realtime-voice-agents 2.2.0 → 2.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +35 -1
- package/dist/{InMemorySessionStore-B5_rq61L.d.cts → InMemorySessionStore-CRBmmCA6.d.cts} +10 -6
- package/dist/{InMemorySessionStore-B5_rq61L.d.mts → InMemorySessionStore-CRBmmCA6.d.mts} +10 -6
- package/dist/index.cjs +236 -9
- package/dist/index.d.cts +172 -2
- package/dist/index.d.mts +172 -2
- package/dist/index.mjs +232 -10
- package/dist/store.d.cts +2 -2
- package/dist/store.d.mts +2 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -184,6 +184,10 @@ const receptionist = new Agent({ name: 'Receptionist', instructions: '…', hand
|
|
|
184
184
|
|
|
185
185
|
Each agent in `handoffs` becomes a `transfer_to_<id>` tool. On handoff the session settles the function call, swaps instructions + tools (`session.update` on OpenAI/xAI; close-and-reopen with context carry on Gemini), and triggers a natural continuation — the caller never hears a seam. Also available programmatically: `session.handoffTo('billing')`. Cycles are fine (billing can hand back).
|
|
186
186
|
|
|
187
|
+
**An agent that just took over cannot transfer again until the caller speaks.** The transfer tool is refused (`agent.handoff.blocked` fires, the model is told why, the active agent does not change); a caller turn — speech or a keypad entry — unlocks it. This makes transfer loops structurally impossible rather than merely discouraged: given the same replayed transcript, each incoming agent otherwise re-derives intent, decides the request is somebody else's, and passes it on. The trade-off is that a pure router node costs an extra caller turn, so direct arcs between agents beat hub-and-spoke. `session.handoffTo()` is host intent and bypasses the lock (it still arms it for the agent it installs).
|
|
188
|
+
|
|
189
|
+
The context an incoming agent receives is attributed, not flat: each replayed line names the agent that said it, and completed transfers appear as `[transfer] A -> B (reason: …)` lines — so it can see what was already answered and already routed.
|
|
190
|
+
|
|
187
191
|
## Built-in call controls
|
|
188
192
|
|
|
189
193
|
```ts
|
|
@@ -270,9 +274,38 @@ session: { greeting: { mode: 'agent-initiates',
|
|
|
270
274
|
|
|
271
275
|
Bundled presets (all synthesized, license-free, seamless loops): `elevator-jazz`, `lofi`, `keyboard-typing`, `thinking-hum`, `ringing` — or `{ custom: bufferOrPath }` with your own 8 kHz μ-law. Drift-corrected 20 ms pacing, refcounted across concurrent tools, ~1 s start delay so fast tools stay silent, fade in/out, 60 s failsafe, and instant preemption when real speech arrives. Manual control: `session.playBackgroundAudio('lofi')` / `stopBackgroundAudio()`.
|
|
272
276
|
|
|
277
|
+
## Keypad input (DTMF, opt-in)
|
|
278
|
+
|
|
279
|
+
Callers type an ID, a phone number, a confirmation code — Twilio delivers each key as its own `dtmf` frame, ~1s apart, and a model that sees ten fragments answers "I didn't get that" ten times. `keypad` turns keypresses into one entry: digits buffer, `#` submits, `*` clears, `maxDigits` auto-submits, 4s without a key flushes what's there (so the agent can say "that's only 7 digits — again, please"). Each keypress stops the agent mid-sentence (typing means "I'm answering"), and the entry reaches the model as a **user** turn — `[keypad] I typed on my phone keypad: 0541234567 — 10 digits. Digit by digit: 0 5 4 …` — that triggers the response answering it. A short note appended to the agent instructions tells the model what `[keypad]` messages are.
|
|
280
|
+
|
|
281
|
+
```ts
|
|
282
|
+
session: {
|
|
283
|
+
keypad: {}, // {} = defaults below
|
|
284
|
+
// maxDigits: 9, // auto-submit at N digits (no # needed)
|
|
285
|
+
// interDigitTimeoutMs: 4000, submitKey: '#', clearKey: '*',
|
|
286
|
+
// interruptOnKeypress: true, // false: the agent keeps talking while the caller types
|
|
287
|
+
// message: (entry) => string | false, // wording of the injected user turn; false = events only
|
|
288
|
+
// clearMessage: string | false, // what the model hears on *; false = nothing
|
|
289
|
+
// instructions: string | false, // the appended note; false = your prompt says it
|
|
290
|
+
}
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
Observe or take over with events and the `session.keypad` handle (`digits`, `clear()`, `submit()`):
|
|
294
|
+
|
|
295
|
+
```ts
|
|
296
|
+
session.on('keypad.entry', ({ digits, reason }) => { /* reason: 'submit' | 'timeout' | 'maxDigits' */ });
|
|
297
|
+
session.on('keypad.cleared', ({ discarded }) => { /* caller pressed * */ });
|
|
298
|
+
// Raw keypresses still fire per key — AFTER the collector consumed them, so the handle is current:
|
|
299
|
+
session.on('dtmf', ({ digit }) => {
|
|
300
|
+
if (digit === '0' && session.keypad.digits === '0') { session.keypad.clear(); void session.transferTo(OPERATOR); }
|
|
301
|
+
});
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
`message: false` keeps the collection and events but injects nothing — validate the entry yourself and `session.sendText(..., { role: 'user', triggerResponse: true })` what the model should hear. Role `user`, not `system`: a trailing system item is skipped by the response it triggers and only lands one response later (field-tested on xAI). Without `keypad` configured nothing changes: raw `dtmf` events only, as before. Letters A–D are ignored; the buffer dies with the call.
|
|
305
|
+
|
|
273
306
|
## Events (session)
|
|
274
307
|
|
|
275
|
-
`call.started/ended/failed` · `provider.connected/fallback/reconnecting/reconnected/closed` · `agent.speech.started/ended` (generation) · **`playback.started/finished/interrupted`** (what the caller heard, mark-confirmed) · `user.speech.started/ended` · `transcript.user/agent` · `tool.started/completed/failed` · `tool.approval.required` · `agent.handoff` · `interruption` / `interruption.blocked` · `vad.suggestion` / `vad.adjusted` (noise-adaptive VAD) · `background_audio.started/stopped` · `dtmf` · `usage.updated` · `error`.
|
|
308
|
+
`call.started/ended/failed` · `provider.connected/fallback/reconnecting/reconnected/closed` · `agent.speech.started/ended` (generation) · **`playback.started/finished/interrupted`** (what the caller heard, mark-confirmed) · `user.speech.started/ended` · `transcript.user/agent` · `tool.started/completed/failed` · `tool.approval.required` · `agent.handoff` / `agent.handoff.blocked` · `interruption` / `interruption.blocked` · `vad.suggestion` / `vad.adjusted` (noise-adaptive VAD) · `background_audio.started/stopped` · `dtmf` (raw keypress) · `keypad.entry` / `keypad.cleared` (keypad input) · `usage.updated` · `error`.
|
|
276
309
|
|
|
277
310
|
```ts
|
|
278
311
|
bridge.on('session.started', (session) => {
|
|
@@ -300,6 +333,7 @@ session: {
|
|
|
300
333
|
hangup: { markTimeoutMs: 7000 }, // goodbye watchdog
|
|
301
334
|
vad: undefined, // normalized VAD, mapped per provider
|
|
302
335
|
noiseAdaptiveVad: undefined, // opt-in noise → VAD escalation ({} enables; see its section)
|
|
336
|
+
keypad: undefined, // opt-in DTMF → one user turn per entry ({} enables; see its section)
|
|
303
337
|
toolResultDelivery: 'afterPlayback', // or 'immediate'
|
|
304
338
|
toolBackgroundAudio: undefined, // default hold audio for tools
|
|
305
339
|
handoffVoicePolicy: 'keep', // or 'reconnect' to switch voices
|
|
@@ -9,6 +9,14 @@ interface TranscriptEntry {
|
|
|
9
9
|
/** The caller cut this utterance off mid-playback. */
|
|
10
10
|
interrupted?: boolean;
|
|
11
11
|
}
|
|
12
|
+
/** One completed transfer, as recorded in the session's handoff history. */
|
|
13
|
+
interface HandoffRecord {
|
|
14
|
+
from: string;
|
|
15
|
+
to: string;
|
|
16
|
+
/** Milliseconds since call start. */
|
|
17
|
+
atMs: number;
|
|
18
|
+
reason?: string;
|
|
19
|
+
}
|
|
12
20
|
//#endregion
|
|
13
21
|
//#region src/session/usage.d.ts
|
|
14
22
|
/** Accumulated token consumption for a call, normalized across providers. */
|
|
@@ -40,11 +48,7 @@ interface CallSnapshot {
|
|
|
40
48
|
transcript: TranscriptEntry[];
|
|
41
49
|
usage: UsageInfo;
|
|
42
50
|
context: Record<string, unknown>;
|
|
43
|
-
handoffHistory:
|
|
44
|
-
from: string;
|
|
45
|
-
to: string;
|
|
46
|
-
atMs: number;
|
|
47
|
-
}>;
|
|
51
|
+
handoffHistory: HandoffRecord[];
|
|
48
52
|
/** Gemini session-resumption handle, when the provider supplies one. */
|
|
49
53
|
resumptionHandle?: string;
|
|
50
54
|
startedAtMs: number;
|
|
@@ -80,4 +84,4 @@ declare class InMemorySessionStore implements SessionStore {
|
|
|
80
84
|
list(): CallSnapshot[];
|
|
81
85
|
}
|
|
82
86
|
//#endregion
|
|
83
|
-
export { emptyUsage as a, UsageInfo as i, SessionStore as n,
|
|
87
|
+
export { emptyUsage as a, UsageInfo as i, SessionStore as n, HandoffRecord as o, CallSnapshot as r, TranscriptEntry as s, InMemorySessionStore as t };
|
|
@@ -9,6 +9,14 @@ interface TranscriptEntry {
|
|
|
9
9
|
/** The caller cut this utterance off mid-playback. */
|
|
10
10
|
interrupted?: boolean;
|
|
11
11
|
}
|
|
12
|
+
/** One completed transfer, as recorded in the session's handoff history. */
|
|
13
|
+
interface HandoffRecord {
|
|
14
|
+
from: string;
|
|
15
|
+
to: string;
|
|
16
|
+
/** Milliseconds since call start. */
|
|
17
|
+
atMs: number;
|
|
18
|
+
reason?: string;
|
|
19
|
+
}
|
|
12
20
|
//#endregion
|
|
13
21
|
//#region src/session/usage.d.ts
|
|
14
22
|
/** Accumulated token consumption for a call, normalized across providers. */
|
|
@@ -40,11 +48,7 @@ interface CallSnapshot {
|
|
|
40
48
|
transcript: TranscriptEntry[];
|
|
41
49
|
usage: UsageInfo;
|
|
42
50
|
context: Record<string, unknown>;
|
|
43
|
-
handoffHistory:
|
|
44
|
-
from: string;
|
|
45
|
-
to: string;
|
|
46
|
-
atMs: number;
|
|
47
|
-
}>;
|
|
51
|
+
handoffHistory: HandoffRecord[];
|
|
48
52
|
/** Gemini session-resumption handle, when the provider supplies one. */
|
|
49
53
|
resumptionHandle?: string;
|
|
50
54
|
startedAtMs: number;
|
|
@@ -80,4 +84,4 @@ declare class InMemorySessionStore implements SessionStore {
|
|
|
80
84
|
list(): CallSnapshot[];
|
|
81
85
|
}
|
|
82
86
|
//#endregion
|
|
83
|
-
export { emptyUsage as a, UsageInfo as i, SessionStore as n,
|
|
87
|
+
export { emptyUsage as a, UsageInfo as i, SessionStore as n, HandoffRecord as o, CallSnapshot as r, TranscriptEntry as s, InMemorySessionStore as t };
|
package/dist/index.cjs
CHANGED
|
@@ -193,6 +193,105 @@ function createHandoffTool(target) {
|
|
|
193
193
|
});
|
|
194
194
|
}
|
|
195
195
|
//#endregion
|
|
196
|
+
//#region src/dtmf/KeypadCollector.ts
|
|
197
|
+
const DEFAULT_KEYPAD_OPTIONS = {
|
|
198
|
+
submitKey: "#",
|
|
199
|
+
clearKey: "*",
|
|
200
|
+
interDigitTimeoutMs: 4e3,
|
|
201
|
+
interruptOnKeypress: true
|
|
202
|
+
};
|
|
203
|
+
/**
|
|
204
|
+
* Appended to the agent instructions when keypad input is enabled. Short and
|
|
205
|
+
* neutral on purpose: what the messages are, not how to run the dialog.
|
|
206
|
+
*/
|
|
207
|
+
const DEFAULT_KEYPAD_INSTRUCTIONS = "Keypad input: the caller may type on their phone keypad instead of speaking. Keypresses arrive as a user message starting with \"[keypad]\" that contains the typed digits — treat it as the caller's answer.";
|
|
208
|
+
/** Default text of the user turn injected for a completed entry. */
|
|
209
|
+
function defaultKeypadMessage(entry) {
|
|
210
|
+
const { digits } = entry;
|
|
211
|
+
return `[keypad] I typed on my phone keypad: ${digits} — ${digits.length} digit${digits.length === 1 ? "" : "s"}. Digit by digit: ${[...digits].join(" ")}`;
|
|
212
|
+
}
|
|
213
|
+
/** Default text of the user turn injected when the caller presses the clear key. */
|
|
214
|
+
const DEFAULT_KEYPAD_CLEAR_MESSAGE = "[keypad] I pressed star — I want to start over and retype from the beginning.";
|
|
215
|
+
var KeypadCollector = class {
|
|
216
|
+
submitKey;
|
|
217
|
+
clearKey;
|
|
218
|
+
interDigitTimeoutMs;
|
|
219
|
+
maxDigits;
|
|
220
|
+
hooks;
|
|
221
|
+
buffer = "";
|
|
222
|
+
timer = null;
|
|
223
|
+
disposed = false;
|
|
224
|
+
constructor(options, hooks) {
|
|
225
|
+
this.submitKey = options.submitKey ?? DEFAULT_KEYPAD_OPTIONS.submitKey;
|
|
226
|
+
this.clearKey = options.clearKey ?? DEFAULT_KEYPAD_OPTIONS.clearKey;
|
|
227
|
+
this.interDigitTimeoutMs = options.interDigitTimeoutMs ?? DEFAULT_KEYPAD_OPTIONS.interDigitTimeoutMs;
|
|
228
|
+
this.maxDigits = options.maxDigits !== void 0 && options.maxDigits > 0 ? options.maxDigits : void 0;
|
|
229
|
+
this.hooks = hooks;
|
|
230
|
+
}
|
|
231
|
+
get digits() {
|
|
232
|
+
return this.buffer;
|
|
233
|
+
}
|
|
234
|
+
/** Feed one keypress (a Twilio `dtmf` frame). Returns what the key meant. */
|
|
235
|
+
press(key) {
|
|
236
|
+
if (this.disposed) return "ignored";
|
|
237
|
+
if (key === this.submitKey) {
|
|
238
|
+
this.cancelTimer();
|
|
239
|
+
this.complete("submit");
|
|
240
|
+
return "submit";
|
|
241
|
+
}
|
|
242
|
+
if (key === this.clearKey) {
|
|
243
|
+
this.cancelTimer();
|
|
244
|
+
const discarded = this.buffer;
|
|
245
|
+
this.buffer = "";
|
|
246
|
+
this.hooks.onClear({ discarded });
|
|
247
|
+
return "clear";
|
|
248
|
+
}
|
|
249
|
+
if (!/^[0-9]$/.test(key)) return "ignored";
|
|
250
|
+
this.cancelTimer();
|
|
251
|
+
this.buffer += key;
|
|
252
|
+
if (this.maxDigits !== void 0 && this.buffer.length >= this.maxDigits) this.complete("maxDigits");
|
|
253
|
+
else this.armTimer();
|
|
254
|
+
return "digit";
|
|
255
|
+
}
|
|
256
|
+
clear() {
|
|
257
|
+
this.cancelTimer();
|
|
258
|
+
this.buffer = "";
|
|
259
|
+
}
|
|
260
|
+
submit() {
|
|
261
|
+
if (this.disposed) return;
|
|
262
|
+
this.cancelTimer();
|
|
263
|
+
this.complete("submit");
|
|
264
|
+
}
|
|
265
|
+
/** Release the timer; pending digits are dropped (the call is over). */
|
|
266
|
+
dispose() {
|
|
267
|
+
this.disposed = true;
|
|
268
|
+
this.cancelTimer();
|
|
269
|
+
this.buffer = "";
|
|
270
|
+
}
|
|
271
|
+
complete(reason) {
|
|
272
|
+
if (!this.buffer) return;
|
|
273
|
+
const digits = this.buffer;
|
|
274
|
+
this.buffer = "";
|
|
275
|
+
this.hooks.onEntry({
|
|
276
|
+
digits,
|
|
277
|
+
reason
|
|
278
|
+
});
|
|
279
|
+
}
|
|
280
|
+
armTimer() {
|
|
281
|
+
this.timer = setTimeout(() => {
|
|
282
|
+
this.timer = null;
|
|
283
|
+
this.complete("timeout");
|
|
284
|
+
}, this.interDigitTimeoutMs);
|
|
285
|
+
this.timer.unref?.();
|
|
286
|
+
}
|
|
287
|
+
cancelTimer() {
|
|
288
|
+
if (this.timer) {
|
|
289
|
+
clearTimeout(this.timer);
|
|
290
|
+
this.timer = null;
|
|
291
|
+
}
|
|
292
|
+
}
|
|
293
|
+
};
|
|
294
|
+
//#endregion
|
|
196
295
|
//#region src/interruption/InterruptionController.ts
|
|
197
296
|
var InterruptionController = class {
|
|
198
297
|
settings;
|
|
@@ -701,9 +800,34 @@ function delayForAttempt(policy, attempt, random = Math.random) {
|
|
|
701
800
|
}
|
|
702
801
|
//#endregion
|
|
703
802
|
//#region src/session/transcript.ts
|
|
704
|
-
/**
|
|
705
|
-
|
|
706
|
-
|
|
803
|
+
/**
|
|
804
|
+
* Render a transcript for history re-injection after reconnect/handoff.
|
|
805
|
+
*
|
|
806
|
+
* Agent lines are attributed to the agent that said them and completed
|
|
807
|
+
* transfers are interleaved as `[transfer]` lines: an incoming agent handed a
|
|
808
|
+
* flat `Agent:` dialogue re-derives intent from scratch, decides the request
|
|
809
|
+
* belongs to somebody else, and transfers on — agents ping-ponging with no
|
|
810
|
+
* caller turn between them (field bug, Aug 2026). Replaying WHO said what and
|
|
811
|
+
* WHAT was already routed is what stops the second lap.
|
|
812
|
+
*/
|
|
813
|
+
function formatTranscriptForInjection(entries, options = {}) {
|
|
814
|
+
const { maxTurns = 30, agentNames, handoffs = [] } = options;
|
|
815
|
+
const recent = entries.slice(-maxTurns);
|
|
816
|
+
if (recent.length === 0) return "";
|
|
817
|
+
const nameOf = (agentId) => (agentId ? agentNames?.get(agentId) : void 0) ?? agentId ?? "Agent";
|
|
818
|
+
const windowStartMs = recent[0].timestampMs;
|
|
819
|
+
const lines = recent.map((entry) => ({
|
|
820
|
+
atMs: entry.timestampMs,
|
|
821
|
+
text: entry.role === "user" ? `Caller: ${entry.text}` : `${nameOf(entry.agentId)}: ${entry.text}`
|
|
822
|
+
}));
|
|
823
|
+
for (const handoff of handoffs) {
|
|
824
|
+
if (handoff.atMs < windowStartMs) continue;
|
|
825
|
+
lines.push({
|
|
826
|
+
atMs: handoff.atMs,
|
|
827
|
+
text: `[transfer] ${nameOf(handoff.from)} -> ${nameOf(handoff.to)}` + (handoff.reason ? ` (reason: ${handoff.reason})` : "")
|
|
828
|
+
});
|
|
829
|
+
}
|
|
830
|
+
return lines.sort((a, b) => a.atMs - b.atMs).map((line) => line.text).join("\n");
|
|
707
831
|
}
|
|
708
832
|
//#endregion
|
|
709
833
|
//#region src/session/usage.ts
|
|
@@ -893,11 +1017,17 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
893
1017
|
streamSid;
|
|
894
1018
|
callInfo;
|
|
895
1019
|
context;
|
|
1020
|
+
/**
|
|
1021
|
+
* Keypad (DTMF) input handle: digits buffered so far, `clear()`, `submit()`.
|
|
1022
|
+
* Inert (empty, no-ops) unless the `keypad` session option is configured.
|
|
1023
|
+
*/
|
|
1024
|
+
keypad;
|
|
896
1025
|
stateValue = "connecting";
|
|
897
1026
|
deps;
|
|
898
1027
|
log;
|
|
899
1028
|
tracker = new PlaybackTracker();
|
|
900
1029
|
interruptions;
|
|
1030
|
+
keypadCollector;
|
|
901
1031
|
usageAccumulator = new UsageAccumulator();
|
|
902
1032
|
toolQueue = new ToolResultQueue();
|
|
903
1033
|
transcriptEntries = [];
|
|
@@ -944,6 +1074,15 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
944
1074
|
agents;
|
|
945
1075
|
handoffHistory = [];
|
|
946
1076
|
handoffInProgress = false;
|
|
1077
|
+
/**
|
|
1078
|
+
* An agent that just took over cannot transfer again until the caller has
|
|
1079
|
+
* spoken. Without it every incoming agent re-derives intent from the same
|
|
1080
|
+
* replayed transcript, decides the request is not its own, and transfers on
|
|
1081
|
+
* — agents ping-ponging with no caller turn between them (field bug, Aug
|
|
1082
|
+
* 2026). Programmatic `handoffTo()` is host intent and bypasses the lock,
|
|
1083
|
+
* but still arms it for the agent it installs.
|
|
1084
|
+
*/
|
|
1085
|
+
handoffLockedUntilCallerTurn = false;
|
|
947
1086
|
/** Pre-synthesized greeting playout state. */
|
|
948
1087
|
pregreeting = null;
|
|
949
1088
|
/** Noise-adaptive VAD (opt-in); built after connect from the provider's ACKed config. */
|
|
@@ -971,6 +1110,18 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
971
1110
|
this.activeAgentValue = deps.agent;
|
|
972
1111
|
this.context = new SessionContext(deps.options.context);
|
|
973
1112
|
this.interruptions = new InterruptionController(deps.options.interruptions);
|
|
1113
|
+
this.keypadCollector = deps.options.keypad ? new KeypadCollector(deps.options.keypad, {
|
|
1114
|
+
onEntry: (entry) => this.onKeypadEntry(entry),
|
|
1115
|
+
onClear: (info) => this.onKeypadCleared(info)
|
|
1116
|
+
}) : null;
|
|
1117
|
+
const collector = this.keypadCollector;
|
|
1118
|
+
this.keypad = {
|
|
1119
|
+
get digits() {
|
|
1120
|
+
return collector?.digits ?? "";
|
|
1121
|
+
},
|
|
1122
|
+
clear: () => collector?.clear(),
|
|
1123
|
+
submit: () => collector?.submit()
|
|
1124
|
+
};
|
|
974
1125
|
const params = deps.start.start.customParameters ?? {};
|
|
975
1126
|
this.callInfo = {
|
|
976
1127
|
direction: params.direction === "outbound" ? "outbound" : "inbound",
|
|
@@ -1212,9 +1363,16 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
1212
1363
|
}
|
|
1213
1364
|
return tools;
|
|
1214
1365
|
}
|
|
1366
|
+
/** Agent instructions plus the kit's own notes that must survive handoffs (keypad). */
|
|
1367
|
+
composeInstructions() {
|
|
1368
|
+
const base = this.activeAgentValue.resolveInstructions(this.context);
|
|
1369
|
+
const keypad = this.deps.options.keypad;
|
|
1370
|
+
if (!keypad || keypad.instructions === false) return base;
|
|
1371
|
+
return `${base}\n\n${keypad.instructions ?? "Keypad input: the caller may type on their phone keypad instead of speaking. Keypresses arrive as a user message starting with \"[keypad]\" that contains the typed digits — treat it as the caller's answer."}`;
|
|
1372
|
+
}
|
|
1215
1373
|
buildProviderInit() {
|
|
1216
1374
|
return {
|
|
1217
|
-
instructions: this.
|
|
1375
|
+
instructions: this.composeInstructions() + (this.pregreeting ? `\n\nYou already opened the call by saying: "${this.pregreeting.text}". Do not greet again — continue the conversation from there.` : ""),
|
|
1218
1376
|
voice: this.activeAgentValue.voice,
|
|
1219
1377
|
vad: this.vadOverride !== void 0 ? this.vadOverride : this.deps.options.vad,
|
|
1220
1378
|
bridgeOwnsInterruptions: true,
|
|
@@ -1237,7 +1395,9 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
1237
1395
|
transport.on("dtmf", (event) => {
|
|
1238
1396
|
this.clearIdleTimer();
|
|
1239
1397
|
this.nudgeCount = 0;
|
|
1240
|
-
|
|
1398
|
+
const digit = event.dtmf.digit;
|
|
1399
|
+
this.handleKeypress(digit);
|
|
1400
|
+
this.emit("dtmf", { digit });
|
|
1241
1401
|
});
|
|
1242
1402
|
transport.on("stop", () => void this.teardown("caller-hangup"));
|
|
1243
1403
|
transport.on("close", () => void this.teardown("caller-hangup"));
|
|
@@ -1306,11 +1466,13 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
1306
1466
|
};
|
|
1307
1467
|
this.transcriptEntries.push(entry);
|
|
1308
1468
|
this.nudgeCount = 0;
|
|
1469
|
+
this.handoffLockedUntilCallerTurn = false;
|
|
1309
1470
|
this.emit("transcript.user", entry);
|
|
1310
1471
|
});
|
|
1311
1472
|
provider.on("userSpeechStarted", () => {
|
|
1312
1473
|
this.userSpeechActive = true;
|
|
1313
1474
|
this.userSpeechStartedAtMs = Date.now();
|
|
1475
|
+
this.handoffLockedUntilCallerTurn = false;
|
|
1314
1476
|
this.clearIdleTimer();
|
|
1315
1477
|
this.nudgeCount = 0;
|
|
1316
1478
|
this.emit("user.speech.started");
|
|
@@ -1606,6 +1768,17 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
1606
1768
|
this.deliverToolResult(call.id, { error: `unknown agent "${directive.targetAgentId}"` });
|
|
1607
1769
|
return;
|
|
1608
1770
|
}
|
|
1771
|
+
if (this.handoffLockedUntilCallerTurn) {
|
|
1772
|
+
this.rejectHandoff(target, directive.reason, call.id);
|
|
1773
|
+
this.emit("tool.completed", {
|
|
1774
|
+
...baseInfo,
|
|
1775
|
+
strategy: tool.strategy,
|
|
1776
|
+
input,
|
|
1777
|
+
result: { handoffBlocked: target.id },
|
|
1778
|
+
durationMs: Date.now() - started
|
|
1779
|
+
});
|
|
1780
|
+
return;
|
|
1781
|
+
}
|
|
1609
1782
|
this.deliverToolResult(call.id, {
|
|
1610
1783
|
status: "transferring_conversation",
|
|
1611
1784
|
to: target.name
|
|
@@ -1989,29 +2162,53 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
1989
2162
|
/** After a reconnect the provider session is blank — restore conversational context. */
|
|
1990
2163
|
reinjectHistory(provider) {
|
|
1991
2164
|
if (this.transcriptEntries.length === 0) return;
|
|
1992
|
-
const summary = formatTranscriptForInjection(this.transcriptEntries
|
|
1993
|
-
|
|
2165
|
+
const summary = formatTranscriptForInjection(this.transcriptEntries, {
|
|
2166
|
+
agentNames: new Map([...this.agents].map(([id, agent]) => [id, agent.name])),
|
|
2167
|
+
handoffs: this.handoffHistory
|
|
2168
|
+
});
|
|
2169
|
+
provider.sendText(`Context: this phone call reconnected mid-conversation. You are ${this.activeAgentValue.name}. Each line below names who said it; \`[transfer]\` lines are routing that ALREADY happened.\n${summary}\nAnything already answered or already routed above must not be routed again. Continue naturally from where it left off; do not greet again.`, {
|
|
1994
2170
|
role: "system",
|
|
1995
2171
|
triggerResponse: false
|
|
1996
2172
|
});
|
|
1997
2173
|
}
|
|
2174
|
+
/**
|
|
2175
|
+
* Refuse a model-initiated transfer from an agent the caller has not spoken
|
|
2176
|
+
* to yet. The refusal must reach the model as the tool result, or it simply
|
|
2177
|
+
* calls the same transfer tool again on its next turn.
|
|
2178
|
+
*/
|
|
2179
|
+
rejectHandoff(target, reason, callId) {
|
|
2180
|
+
const from = this.activeAgentValue;
|
|
2181
|
+
this.log.warn(`blocked transfer ${from.id} -> ${target.id}: the caller has not spoken since the last one`);
|
|
2182
|
+
this.deliverToolResult(callId, {
|
|
2183
|
+
error: "transfer_rejected",
|
|
2184
|
+
message: "You just took over this call and the caller has not spoken since. Do not transfer again yet — handle their request yourself, or ask them what they need. You may transfer once they reply."
|
|
2185
|
+
});
|
|
2186
|
+
this.emit("agent.handoff.blocked", {
|
|
2187
|
+
from,
|
|
2188
|
+
to: target,
|
|
2189
|
+
reason,
|
|
2190
|
+
cause: "no-caller-turn"
|
|
2191
|
+
});
|
|
2192
|
+
}
|
|
1998
2193
|
async performHandoff(target, reason) {
|
|
1999
2194
|
if (this.stateValue !== "active" || !this.provider) return;
|
|
2000
2195
|
if (target.id === this.activeAgentValue.id) return;
|
|
2001
2196
|
const from = this.activeAgentValue;
|
|
2002
2197
|
this.activeAgentValue = target;
|
|
2198
|
+
this.handoffLockedUntilCallerTurn = true;
|
|
2003
2199
|
this.toolset = this.buildToolset();
|
|
2004
2200
|
this.handoffHistory.push({
|
|
2005
2201
|
from: from.id,
|
|
2006
2202
|
to: target.id,
|
|
2007
|
-
atMs: Date.now() - this.startedAtMs
|
|
2203
|
+
atMs: Date.now() - this.startedAtMs,
|
|
2204
|
+
...reason ? { reason } : {}
|
|
2008
2205
|
});
|
|
2009
2206
|
const continueInstruction = `You are now ${target.name}. Continue the SAME phone conversation naturally — acknowledge the caller and take over; do not restart with a cold greeting.` + (reason ? ` Transfer context: ${reason}.` : "");
|
|
2010
2207
|
const voiceChanges = target.voice !== void 0 && target.voice !== from.voice;
|
|
2011
2208
|
if (!(!this.provider.capabilities.sessionUpdate || voiceChanges && !this.provider.capabilities.voiceChangeMidSession && this.deps.options.handoffVoicePolicy === "reconnect")) {
|
|
2012
2209
|
if (voiceChanges && !this.provider.capabilities.voiceChangeMidSession) this.log.warn(`agent "${target.id}" declares voice "${target.voice}" but the provider cannot change voice mid-session — keeping the current voice (set handoffVoicePolicy: 'reconnect' to switch)`);
|
|
2013
2210
|
await this.provider.updateSession({
|
|
2014
|
-
instructions: this.
|
|
2211
|
+
instructions: this.composeInstructions(),
|
|
2015
2212
|
tools: this.buildProviderInit().tools,
|
|
2016
2213
|
providerOptions: target.providerOptions
|
|
2017
2214
|
});
|
|
@@ -2087,6 +2284,30 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
2087
2284
|
this.transcriptEntries.push(entry);
|
|
2088
2285
|
this.emit("transcript.agent", entry);
|
|
2089
2286
|
}
|
|
2287
|
+
handleKeypress(key) {
|
|
2288
|
+
if (!this.keypadCollector) return;
|
|
2289
|
+
if (this.deps.options.keypad?.interruptOnKeypress !== false) this.interrupt();
|
|
2290
|
+
this.keypadCollector.press(key);
|
|
2291
|
+
}
|
|
2292
|
+
onKeypadEntry(entry) {
|
|
2293
|
+
this.handoffLockedUntilCallerTurn = false;
|
|
2294
|
+
this.emit("keypad.entry", entry);
|
|
2295
|
+
const message = this.deps.options.keypad?.message;
|
|
2296
|
+
if (message === false) return;
|
|
2297
|
+
this.provider?.sendText((message ?? defaultKeypadMessage)(entry), {
|
|
2298
|
+
role: "user",
|
|
2299
|
+
triggerResponse: true
|
|
2300
|
+
});
|
|
2301
|
+
}
|
|
2302
|
+
onKeypadCleared(info) {
|
|
2303
|
+
this.emit("keypad.cleared", info);
|
|
2304
|
+
const clearMessage = this.deps.options.keypad?.clearMessage;
|
|
2305
|
+
if (clearMessage === false) return;
|
|
2306
|
+
this.provider?.sendText(clearMessage ?? "[keypad] I pressed star — I want to start over and retype from the beginning.", {
|
|
2307
|
+
role: "user",
|
|
2308
|
+
triggerResponse: true
|
|
2309
|
+
});
|
|
2310
|
+
}
|
|
2090
2311
|
/** Armed whenever the agent goes quiet and we're waiting on the caller. */
|
|
2091
2312
|
armIdleTimer() {
|
|
2092
2313
|
const idle = this.deps.options.idle;
|
|
@@ -2146,6 +2367,7 @@ var CallSession = class extends require_events.TypedEmitter {
|
|
|
2146
2367
|
for (const timer of this.timers) clearTimeout(timer);
|
|
2147
2368
|
this.timers.clear();
|
|
2148
2369
|
this.clearIdleTimer();
|
|
2370
|
+
this.keypadCollector?.dispose();
|
|
2149
2371
|
this.bgAudio.stop({ immediate: true });
|
|
2150
2372
|
for (const controller of this.runningTools.values()) controller.abort(/* @__PURE__ */ new Error("call ended"));
|
|
2151
2373
|
this.runningTools.clear();
|
|
@@ -2456,10 +2678,14 @@ async function captureGreetingAudio(options) {
|
|
|
2456
2678
|
exports.Agent = Agent;
|
|
2457
2679
|
exports.BaseRealtimeProvider = require_BaseRealtimeProvider.BaseRealtimeProvider;
|
|
2458
2680
|
exports.CallSession = CallSession;
|
|
2681
|
+
exports.DEFAULT_KEYPAD_CLEAR_MESSAGE = DEFAULT_KEYPAD_CLEAR_MESSAGE;
|
|
2682
|
+
exports.DEFAULT_KEYPAD_INSTRUCTIONS = DEFAULT_KEYPAD_INSTRUCTIONS;
|
|
2683
|
+
exports.DEFAULT_KEYPAD_OPTIONS = DEFAULT_KEYPAD_OPTIONS;
|
|
2459
2684
|
exports.DEFAULT_RECONNECT_POLICY = DEFAULT_RECONNECT_POLICY;
|
|
2460
2685
|
exports.DEFAULT_SESSION_OPTIONS = DEFAULT_SESSION_OPTIONS;
|
|
2461
2686
|
exports.InMemorySessionStore = require_InMemorySessionStore.InMemorySessionStore;
|
|
2462
2687
|
exports.InterruptionController = InterruptionController;
|
|
2688
|
+
exports.KeypadCollector = KeypadCollector;
|
|
2463
2689
|
exports.NoiseAdaptiveVadController = NoiseAdaptiveVadController;
|
|
2464
2690
|
exports.PlaybackTracker = PlaybackTracker;
|
|
2465
2691
|
exports.SessionContext = SessionContext;
|
|
@@ -2473,6 +2699,7 @@ exports.createFinishCallTool = createFinishCallTool;
|
|
|
2473
2699
|
exports.createHandoffTool = createHandoffTool;
|
|
2474
2700
|
exports.createTransferCallTool = createTransferCallTool;
|
|
2475
2701
|
exports.decorateTool = decorateTool;
|
|
2702
|
+
exports.defaultKeypadMessage = defaultKeypadMessage;
|
|
2476
2703
|
exports.emptyUsage = emptyUsage;
|
|
2477
2704
|
exports.handoffToolName = handoffToolName;
|
|
2478
2705
|
exports.isHandoffDirective = isHandoffDirective;
|