dsh-realtime-agent 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +96 -2
- package/lib/bridge.d.ts +41 -0
- package/lib/bridge.d.ts.map +1 -0
- package/lib/bridge.js +81 -0
- package/lib/bridge.js.map +1 -0
- package/lib/index.d.ts +102 -0
- package/lib/index.d.ts.map +1 -0
- package/lib/index.js +137 -0
- package/lib/index.js.map +1 -0
- package/lib/tools.d.ts +22 -0
- package/lib/tools.d.ts.map +1 -0
- package/lib/tools.js +88 -0
- package/lib/tools.js.map +1 -0
- package/lib/transcript.d.ts +35 -0
- package/lib/transcript.d.ts.map +1 -0
- package/lib/transcript.js +57 -0
- package/lib/transcript.js.map +1 -0
- package/lib/types.d.ts +71 -0
- package/lib/types.d.ts.map +1 -0
- package/lib/types.js +8 -0
- package/lib/types.js.map +1 -0
- package/package.json +58 -3
- package/src/bridge.ts +101 -0
- package/src/index.ts +179 -0
- package/src/tools.ts +108 -0
- package/src/transcript.ts +65 -0
- package/src/types.ts +74 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Travis Driessen
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,97 @@
|
|
|
1
|
-
#
|
|
1
|
+
# dsh-realtime-agent
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**The consumer for the [`dsh-realtime`](../realtime) seam: it answers what the voice model delegates.**
|
|
4
|
+
|
|
5
|
+
A voice model that can hand work to the host is only useful if something picks it up. This plugin
|
|
6
|
+
holds a live session, records the conversation, and when the model raises a delegation it asks the
|
|
7
|
+
application — and then **either delivers the answer or says plainly that it cannot.**
|
|
8
|
+
|
|
9
|
+
## Why the delegation is the whole point
|
|
10
|
+
|
|
11
|
+
Everything else about realtime voice is transport. A fast handshake and clean audio are table stakes;
|
|
12
|
+
the thing that makes a voice model an *agent* is that it can ask for something in the middle of a
|
|
13
|
+
conversation and keep talking while it waits.
|
|
14
|
+
|
|
15
|
+
The protocol makes that deliberately hard to fudge. `session.delegation.created` carries
|
|
16
|
+
**metadata only** — `{id, target, offsetMs}` — and **no task text**:
|
|
17
|
+
|
|
18
|
+
```ts
|
|
19
|
+
interface RealtimeDelegation {
|
|
20
|
+
id: string
|
|
21
|
+
target: 'client' | 'responses'
|
|
22
|
+
offsetMs: number
|
|
23
|
+
}
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
So intent has to be reconstructed. This plugin carries the conversation it has transcribed alongside
|
|
27
|
+
the delegation, because the consumer knows the conversation and only the application knows its own
|
|
28
|
+
state. Neither can do it alone, which is why the request carries one and is handed to the other.
|
|
29
|
+
|
|
30
|
+
## Answering
|
|
31
|
+
|
|
32
|
+
Listen on the bus. Listeners run in order, and the **first to return a non-empty answer wins** — that
|
|
33
|
+
is Cordis `serial` dispatch, so several responders can be registered and each can decline by
|
|
34
|
+
returning nothing:
|
|
35
|
+
|
|
36
|
+
```ts
|
|
37
|
+
ctx.on('realtime-agent/delegation', async (request) => {
|
|
38
|
+
// request: { id, offsetMs, transcript, sessionId }
|
|
39
|
+
const answer = await lookSomethingUp(request.transcript)
|
|
40
|
+
return answer === undefined ? undefined : { text: answer, mode: 'spoken' }
|
|
41
|
+
})
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
`mode: 'spoken'` sends the text as commentary, which the model **says aloud** — use it when the person
|
|
45
|
+
in the conversation is waiting. `silent` (the default) sends it as thinking, which the model may use
|
|
46
|
+
without announcing it.
|
|
47
|
+
|
|
48
|
+
## Failing closed, out loud
|
|
49
|
+
|
|
50
|
+
Returning nothing is a decline, and it is never silent. If every responder declines, none is
|
|
51
|
+
registered, the responder throws, or the answer does not arrive within `delegationTimeoutMs`, the
|
|
52
|
+
model is told:
|
|
53
|
+
|
|
54
|
+
> *"Sorry — I can't take care of that right now."*
|
|
55
|
+
|
|
56
|
+
That is the seam's first invariant made audible: an unresolvable or erroring delegation **never
|
|
57
|
+
auto-approves**. The two failure modes it rules out are a fabricated answer and a silence the person
|
|
58
|
+
on the other end reads as comprehension.
|
|
59
|
+
|
|
60
|
+
## Configuration
|
|
61
|
+
|
|
62
|
+
```yaml
|
|
63
|
+
- name: dsh-realtime-agent
|
|
64
|
+
config:
|
|
65
|
+
provider: openai-live # the registered realtime route
|
|
66
|
+
model: gpt-live-1
|
|
67
|
+
voice: marin # optional; omitted keeps the provider default
|
|
68
|
+
instructions: '…' # optional; opening instructions for the session
|
|
69
|
+
autoStart: false # mounting must not, by itself, open a socket or spend credit
|
|
70
|
+
delegationTimeoutMs: 10000
|
|
71
|
+
maxTranscriptChars: 6000
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
A session is opened only when `autoStart` is set. `maxTranscriptChars` bounds the transcript carried
|
|
75
|
+
on each request: it is the **oldest** lines that are dropped, and the most recent line is always kept,
|
|
76
|
+
because a buffer that can hold nothing can answer nothing.
|
|
77
|
+
|
|
78
|
+
## How answers are kept deliverable
|
|
79
|
+
|
|
80
|
+
- **A long answer is cut, not rejected.** The seam bounds an append at 2000 characters and *throws*
|
|
81
|
+
past it; a clearly-marked prefix beats losing the answer to an exception raised inside a handler.
|
|
82
|
+
- **A bad responder is a decline.** A synchronous throw, a rejection, or a malformed return all lead
|
|
83
|
+
to the same honest notice rather than to an exception escaping the adapter's callback.
|
|
84
|
+
- **The bound is unref'd.** A timeout that loses the race cannot hold the process open, and there is
|
|
85
|
+
no timer id to clear — so no failure path can leak one.
|
|
86
|
+
- **A closed session is not answered into.** Speaking into a released transport would raise a failure
|
|
87
|
+
from inside a handler, so the handler declines to act instead.
|
|
88
|
+
|
|
89
|
+
## Tested
|
|
90
|
+
|
|
91
|
+
Per-file 100% coverage on all four axes, enforced. The delegation path is covered end to end through
|
|
92
|
+
the real seam, a real `serial` dispatch and a real session — including every way of declining, the
|
|
93
|
+
timeout, a responder that throws, and an answer longer than the seam allows.
|
|
94
|
+
|
|
95
|
+
## Licence
|
|
96
|
+
|
|
97
|
+
MIT.
|
package/lib/bridge.d.ts
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
import { type RealtimeDelegation, type RealtimeSession } from 'dsh-realtime';
|
|
2
|
+
import type { AgentTranscriptLine, DelegationAnswer, DelegationRequest } from './types.ts';
|
|
3
|
+
/**
|
|
4
|
+
* Spoken when a delegation could not be answered.
|
|
5
|
+
*
|
|
6
|
+
* Never silent, and never a fabricated answer. The seam's first invariant is that an unresolvable or
|
|
7
|
+
* erroring delegation never auto-approves; the honest complement is that the model is told so out
|
|
8
|
+
* loud, because the person in the conversation is the one waiting.
|
|
9
|
+
*/
|
|
10
|
+
export declare const UNANSWERED_NOTICE = "Sorry \u2014 I can't take care of that right now.";
|
|
11
|
+
/**
|
|
12
|
+
* Ask the application for an answer.
|
|
13
|
+
*
|
|
14
|
+
* Synchronous and asynchronous responders are both acceptable, and a decline is `undefined`.
|
|
15
|
+
*/
|
|
16
|
+
export interface DelegationAsker {
|
|
17
|
+
(request: DelegationRequest): DelegationAnswer | undefined | Promise<DelegationAnswer | undefined>;
|
|
18
|
+
}
|
|
19
|
+
/**
|
|
20
|
+
* Cut an answer to the seam's append bound.
|
|
21
|
+
*
|
|
22
|
+
* A bound applied here rather than at the append is deliberate: the append would *throw*, and losing a
|
|
23
|
+
* long answer entirely is worse than delivering a marked prefix of it.
|
|
24
|
+
* @param text - the answer.
|
|
25
|
+
* @param maxChars - ceiling, defaulting to the seam's own bound.
|
|
26
|
+
* @returns the text, or a marked prefix of it no longer than `maxChars`.
|
|
27
|
+
*/
|
|
28
|
+
export declare function boundAppend(text: string, maxChars?: number): string;
|
|
29
|
+
/**
|
|
30
|
+
* Answer one delegation, or tell the model plainly that it could not be answered.
|
|
31
|
+
*
|
|
32
|
+
* Dispatched rather than returned: the caller is a session handler, which cannot await.
|
|
33
|
+
* @param session - the live session the delegation arrived on.
|
|
34
|
+
* @param delegation - the delegation, whose `id` is carried back on the answer.
|
|
35
|
+
* @param transcript - the conversation so far, as the responder will see it.
|
|
36
|
+
* @param ask - how to reach a responder.
|
|
37
|
+
* @param timeoutMs - bound on waiting for one.
|
|
38
|
+
* @returns a promise settling once the session has been answered.
|
|
39
|
+
*/
|
|
40
|
+
export declare function answerDelegation(session: Pick<RealtimeSession, 'id' | 'appendCommentary' | 'appendThinking'>, delegation: RealtimeDelegation, transcript: readonly AgentTranscriptLine[], ask: DelegationAsker, timeoutMs: number): Promise<void>;
|
|
41
|
+
//# sourceMappingURL=bridge.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"bridge.d.ts","sourceRoot":"","sources":["../src/bridge.ts"],"names":[],"mappings":"AAAA,OAAO,EAAoB,KAAK,kBAAkB,EAAE,KAAK,eAAe,EAAE,MAAM,cAAc,CAAA;AAC9F,OAAO,KAAK,EAAE,mBAAmB,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAA;AAE1F;;;;;;GAMG;AACH,eAAO,MAAM,iBAAiB,sDAAiD,CAAA;AAK/E;;;;GAIG;AACH,MAAM,WAAW,eAAe;IAC9B,CAAC,OAAO,EAAE,iBAAiB,GAAG,gBAAgB,GAAG,SAAS,GAAG,OAAO,CAAC,gBAAgB,GAAG,SAAS,CAAC,CAAA;CACnG;AAED;;;;;;;;GAQG;AACH,wBAAgB,WAAW,CAAC,IAAI,EAAE,MAAM,EAAE,QAAQ,GAAE,MAAyB,GAAG,MAAM,CAKrF;AAmBD;;;;;;;;;;GAUG;AACH,wBAAsB,gBAAgB,CACpC,OAAO,EAAE,IAAI,CAAC,eAAe,EAAE,IAAI,GAAG,kBAAkB,GAAG,gBAAgB,CAAC,EAC5E,UAAU,EAAE,kBAAkB,EAC9B,UAAU,EAAE,SAAS,mBAAmB,EAAE,EAC1C,GAAG,EAAE,eAAe,EACpB,SAAS,EAAE,MAAM,GAChB,OAAO,CAAC,IAAI,CAAC,CA0Bf"}
|
package/lib/bridge.js
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
import { MAX_APPEND_CHARS } from 'dsh-realtime';
|
|
2
|
+
/**
|
|
3
|
+
* Spoken when a delegation could not be answered.
|
|
4
|
+
*
|
|
5
|
+
* Never silent, and never a fabricated answer. The seam's first invariant is that an unresolvable or
|
|
6
|
+
* erroring delegation never auto-approves; the honest complement is that the model is told so out
|
|
7
|
+
* loud, because the person in the conversation is the one waiting.
|
|
8
|
+
*/
|
|
9
|
+
export const UNANSWERED_NOTICE = "Sorry — I can't take care of that right now.";
|
|
10
|
+
/** Appended to an answer cut to fit the seam's append bound, so a reader knows it is partial. */
|
|
11
|
+
const TRUNCATION_MARKER = ' [truncated]';
|
|
12
|
+
/**
|
|
13
|
+
* Cut an answer to the seam's append bound.
|
|
14
|
+
*
|
|
15
|
+
* A bound applied here rather than at the append is deliberate: the append would *throw*, and losing a
|
|
16
|
+
* long answer entirely is worse than delivering a marked prefix of it.
|
|
17
|
+
* @param text - the answer.
|
|
18
|
+
* @param maxChars - ceiling, defaulting to the seam's own bound.
|
|
19
|
+
* @returns the text, or a marked prefix of it no longer than `maxChars`.
|
|
20
|
+
*/
|
|
21
|
+
export function boundAppend(text, maxChars = MAX_APPEND_CHARS) {
|
|
22
|
+
if (text.length <= maxChars)
|
|
23
|
+
return text;
|
|
24
|
+
// A ceiling shorter than the marker itself can only carry the marker's own prefix.
|
|
25
|
+
if (maxChars <= TRUNCATION_MARKER.length)
|
|
26
|
+
return TRUNCATION_MARKER.slice(0, maxChars);
|
|
27
|
+
return text.slice(0, maxChars - TRUNCATION_MARKER.length) + TRUNCATION_MARKER;
|
|
28
|
+
}
|
|
29
|
+
/**
|
|
30
|
+
* Run the asker, turning any failure into a decline.
|
|
31
|
+
*
|
|
32
|
+
* A responder that throws must not become an exception the adapter's handler sees, and must not be
|
|
33
|
+
* mistaken for an answer: failing to answer and choosing not to answer lead to the same honest notice.
|
|
34
|
+
* @param ask - the asker.
|
|
35
|
+
* @param request - what to ask.
|
|
36
|
+
* @returns the answer, or `undefined` when the asker failed.
|
|
37
|
+
*/
|
|
38
|
+
function settled(ask, request) {
|
|
39
|
+
try {
|
|
40
|
+
return Promise.resolve(ask(request)).catch(() => undefined);
|
|
41
|
+
}
|
|
42
|
+
catch {
|
|
43
|
+
return Promise.resolve(undefined);
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
/**
|
|
47
|
+
* Answer one delegation, or tell the model plainly that it could not be answered.
|
|
48
|
+
*
|
|
49
|
+
* Dispatched rather than returned: the caller is a session handler, which cannot await.
|
|
50
|
+
* @param session - the live session the delegation arrived on.
|
|
51
|
+
* @param delegation - the delegation, whose `id` is carried back on the answer.
|
|
52
|
+
* @param transcript - the conversation so far, as the responder will see it.
|
|
53
|
+
* @param ask - how to reach a responder.
|
|
54
|
+
* @param timeoutMs - bound on waiting for one.
|
|
55
|
+
* @returns a promise settling once the session has been answered.
|
|
56
|
+
*/
|
|
57
|
+
export async function answerDelegation(session, delegation, transcript, ask, timeoutMs) {
|
|
58
|
+
const request = {
|
|
59
|
+
id: delegation.id,
|
|
60
|
+
offsetMs: delegation.offsetMs,
|
|
61
|
+
transcript,
|
|
62
|
+
sessionId: session.id,
|
|
63
|
+
};
|
|
64
|
+
// An unref'd bound, so a timer that loses the race cannot hold the process open — and so there is
|
|
65
|
+
// no id to clear, which means no failure path can leak one.
|
|
66
|
+
const expired = new Promise((resolve) => {
|
|
67
|
+
setTimeout(() => { resolve(undefined); }, timeoutMs).unref();
|
|
68
|
+
});
|
|
69
|
+
const answer = await Promise.race([settled(ask, request), expired]);
|
|
70
|
+
const text = typeof answer?.text === 'string' ? answer.text.trim() : '';
|
|
71
|
+
if (text.length === 0) {
|
|
72
|
+
await session.appendCommentary(UNANSWERED_NOTICE, delegation.id);
|
|
73
|
+
return;
|
|
74
|
+
}
|
|
75
|
+
if (answer?.mode === 'spoken') {
|
|
76
|
+
await session.appendCommentary(boundAppend(text), delegation.id);
|
|
77
|
+
return;
|
|
78
|
+
}
|
|
79
|
+
await session.appendThinking(boundAppend(text), delegation.id);
|
|
80
|
+
}
|
|
81
|
+
//# sourceMappingURL=bridge.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"bridge.js","sourceRoot":"","sources":["../src/bridge.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAiD,MAAM,cAAc,CAAA;AAG9F;;;;;;GAMG;AACH,MAAM,CAAC,MAAM,iBAAiB,GAAG,8CAA8C,CAAA;AAE/E,iGAAiG;AACjG,MAAM,iBAAiB,GAAG,cAAc,CAAA;AAWxC;;;;;;;;GAQG;AACH,MAAM,UAAU,WAAW,CAAC,IAAY,EAAE,QAAQ,GAAW,gBAAgB;IAC3E,IAAI,IAAI,CAAC,MAAM,IAAI,QAAQ;QAAE,OAAO,IAAI,CAAA;IACxC,mFAAmF;IACnF,IAAI,QAAQ,IAAI,iBAAiB,CAAC,MAAM;QAAE,OAAO,iBAAiB,CAAC,KAAK,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAA;IACrF,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,QAAQ,GAAG,iBAAiB,CAAC,MAAM,CAAC,GAAG,iBAAiB,CAAA;AAC/E,CAAC;AAED;;;;;;;;GAQG;AACH,SAAS,OAAO,CAAC,GAAoB,EAAE,OAA0B;IAC/D,IAAI,CAAC;QACH,OAAO,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAA;IAC7D,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,OAAO,CAAC,OAAO,CAAC,SAAS,CAAC,CAAA;IACnC,CAAC;AACH,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,CAAC,KAAK,UAAU,gBAAgB,CACpC,OAA4E,EAC5E,UAA8B,EAC9B,UAA0C,EAC1C,GAAoB,EACpB,SAAiB;IAEjB,MAAM,OAAO,GAAsB;QACjC,EAAE,EAAE,UAAU,CAAC,EAAE;QACjB,QAAQ,EAAE,UAAU,CAAC,QAAQ;QAC7B,UAAU;QACV,SAAS,EAAE,OAAO,CAAC,EAAE;KACtB,CAAA;IAED,kGAAkG;IAClG,4DAA4D;IAC5D,MAAM,OAAO,GAAG,IAAI,OAAO,CAAY,CAAC,OAAO,EAAE,EAAE;QACjD,UAAU,CAAC,GAAG,EAAE,GAAG,OAAO,CAAC,SAAS,CAAC,CAAA,CAAC,CAAC,EAAE,SAAS,CAAC,CAAC,KAAK,EAAE,CAAA;IAC7D,CAAC,CAAC,CAAA;IAEF,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC,OAAO,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,OAAO,CAAC,CAAC,CAAA;IACnE,MAAM,IAAI,GAAG,OAAO,MAAM,EAAE,IAAI,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAA;IAEvE,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACtB,MAAM,OAAO,CAAC,gBAAgB,CAAC,iBAAiB,EAAE,UAAU,CAAC,EAAE,CAAC,CAAA;QAChE,OAAM;IACR,CAAC;IACD,IAAI,MAAM,EAAE,IAAI,KAAK,QAAQ,EAAE,CAAC;QAC9B,MAAM,OAAO,CAAC,gBAAgB,CAAC,WAAW,CAAC,IAAI,CAAC,EAAE,UAAU,CAAC,EAAE,CAAC,CAAA;QAChE,OAAM;IACR,CAAC;IACD,MAAM,OAAO,CAAC,cAAc,CAAC,WAAW,CAAC,IAAI,CAAC,EAAE,UAAU,CAAC,EAAE,CAAC,CAAA;AAChE,CAAC"}
|
package/lib/index.d.ts
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The realtime agent consumer: answers what the voice model delegates.
|
|
3
|
+
*
|
|
4
|
+
* A **function plugin** on the `dsh-realtime` seam. It holds a live session, records the conversation,
|
|
5
|
+
* and when the model raises a delegation it asks the application — over the event bus, with `serial`
|
|
6
|
+
* semantics so the first responder to answer wins — and delivers the result, or says plainly that it
|
|
7
|
+
* cannot be handled.
|
|
8
|
+
*
|
|
9
|
+
* Per the harness convention it named-exports `name` / `inject` / `Config` / `apply` and has **no
|
|
10
|
+
* default export** — adding one makes the Loader discard this plugin's namespace, so the plugin would
|
|
11
|
+
* load and contribute nothing.
|
|
12
|
+
*
|
|
13
|
+
* An application answers by listening:
|
|
14
|
+
*
|
|
15
|
+
* ```ts
|
|
16
|
+
* ctx.on('realtime-agent/delegation', (request) => {
|
|
17
|
+
* return { text: lookItUp(request.transcript), mode: 'spoken' }
|
|
18
|
+
* })
|
|
19
|
+
* ```
|
|
20
|
+
*
|
|
21
|
+
* @module dsh-realtime-agent
|
|
22
|
+
*/
|
|
23
|
+
import Schema from '@deepseek-ai/schemastery';
|
|
24
|
+
import type { Context } from '@deepseek-ai/cordis';
|
|
25
|
+
import type { RealtimeSession, RealtimeSessionHandlers } from 'dsh-realtime';
|
|
26
|
+
import { type DelegationAsker } from './bridge.ts';
|
|
27
|
+
import { TranscriptBuffer } from './transcript.ts';
|
|
28
|
+
import type { DelegationAnswer, DelegationRequest, RealtimeAgentConfig } from './types.ts';
|
|
29
|
+
declare module '@deepseek-ai/cordis' {
|
|
30
|
+
interface Events {
|
|
31
|
+
/**
|
|
32
|
+
* Dispatched when the voice model delegates work, awaited with **serial** semantics: listeners run
|
|
33
|
+
* in order, and the first to return a non-empty answer wins.
|
|
34
|
+
*
|
|
35
|
+
* Return `undefined` to decline and let a later responder try. If every listener declines — or none
|
|
36
|
+
* is registered — the model is told out loud that the request could not be handled.
|
|
37
|
+
*/
|
|
38
|
+
'realtime-agent/delegation'(request: DelegationRequest): DelegationAnswer | undefined | Promise<DelegationAnswer | undefined>;
|
|
39
|
+
/** A session-scoped failure the adapter contained rather than throwing. */
|
|
40
|
+
'realtime-agent/error'(error: Error): void;
|
|
41
|
+
}
|
|
42
|
+
}
|
|
43
|
+
export * from './types.ts';
|
|
44
|
+
export { answerDelegation, boundAppend, UNANSWERED_NOTICE, type DelegationAsker } from './bridge.ts';
|
|
45
|
+
export { TranscriptBuffer } from './transcript.ts';
|
|
46
|
+
export { voiceToolDefinitions, type VoiceToolDeps } from './tools.ts';
|
|
47
|
+
/** Plugin name, as it appears in Loader diagnostics. */
|
|
48
|
+
export declare const name = "realtime-agent";
|
|
49
|
+
/** This plugin drives the realtime seam and publishes tools, so it waits for both. */
|
|
50
|
+
export declare const inject: string[];
|
|
51
|
+
/**
|
|
52
|
+
* Validated configuration.
|
|
53
|
+
*
|
|
54
|
+
* `autoStart` defaults to `false`: mounting this plugin must not, by itself, open a socket or spend
|
|
55
|
+
* credit. `voice` and `instructions` are omitted unless set, so the provider default applies rather
|
|
56
|
+
* than an empty string the provider would reject.
|
|
57
|
+
*/
|
|
58
|
+
export declare const Config: Schema<Schemastery.ObjectS<NoInfer<{
|
|
59
|
+
provider: Schema<string, string, "defined">;
|
|
60
|
+
model: Schema<string, string, "defined">;
|
|
61
|
+
voice: Schema<string, string, "plain">;
|
|
62
|
+
instructions: Schema<string, string, "plain">;
|
|
63
|
+
autoStart: Schema<boolean, boolean, "defined">;
|
|
64
|
+
delegationTimeoutMs: Schema<number, number, "defined">;
|
|
65
|
+
maxTranscriptChars: Schema<number, number, "defined">;
|
|
66
|
+
}>>, Schemastery.ObjectT<NoInfer<{
|
|
67
|
+
provider: Schema<string, string, "defined">;
|
|
68
|
+
model: Schema<string, string, "defined">;
|
|
69
|
+
voice: Schema<string, string, "plain">;
|
|
70
|
+
instructions: Schema<string, string, "plain">;
|
|
71
|
+
autoStart: Schema<boolean, boolean, "defined">;
|
|
72
|
+
delegationTimeoutMs: Schema<number, number, "defined">;
|
|
73
|
+
maxTranscriptChars: Schema<number, number, "defined">;
|
|
74
|
+
}>>, "plain">;
|
|
75
|
+
/** What {@link createHandlers} needs. Injected rather than closed over, so the handlers are testable. */
|
|
76
|
+
export interface HandlerDeps {
|
|
77
|
+
/** The conversation recorded so far. */
|
|
78
|
+
readonly transcript: TranscriptBuffer;
|
|
79
|
+
/** The live session a handler should act on, or `undefined` when there is none. */
|
|
80
|
+
readonly session: () => RealtimeSession | undefined;
|
|
81
|
+
/** How to reach a responder. */
|
|
82
|
+
readonly ask: DelegationAsker;
|
|
83
|
+
/** Bound on waiting for one. */
|
|
84
|
+
readonly timeoutMs: number;
|
|
85
|
+
/** Called when the session ended, so the caller can drop its reference. */
|
|
86
|
+
readonly onClosed: () => void;
|
|
87
|
+
/** Where a session-scoped failure is reported. */
|
|
88
|
+
readonly onSessionError: (error: Error) => void;
|
|
89
|
+
}
|
|
90
|
+
/**
|
|
91
|
+
* Build the session handlers.
|
|
92
|
+
* @param deps - the session, the transcript and the asker.
|
|
93
|
+
* @returns handlers the adapter can invoke for the life of the session.
|
|
94
|
+
*/
|
|
95
|
+
export declare function createHandlers(deps: HandlerDeps): RealtimeSessionHandlers;
|
|
96
|
+
/**
|
|
97
|
+
* Hold a session and bridge what the voice model delegates.
|
|
98
|
+
* @param ctx - the Cordis context, which must already provide the `realtime` service.
|
|
99
|
+
* @param config - validated configuration.
|
|
100
|
+
*/
|
|
101
|
+
export declare function apply(ctx: Context, config: RealtimeAgentConfig): void;
|
|
102
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;GAqBG;AAEH,OAAO,MAAM,MAAM,0BAA0B,CAAA;AAC7C,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAClD,OAAO,KAAK,EAAsB,eAAe,EAAE,uBAAuB,EAAsB,MAAM,cAAc,CAAA;AACpH,OAAO,EAAoB,KAAK,eAAe,EAAE,MAAM,aAAa,CAAA;AAEpE,OAAO,EAAE,gBAAgB,EAAE,MAAM,iBAAiB,CAAA;AAClD,OAAO,KAAK,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,mBAAmB,EAAE,MAAM,YAAY,CAAA;AAE1F,OAAO,QAAQ,qBAAqB,CAAC;IACnC,UAAU,MAAM;QACd;;;;;;WAMG;QACH,2BAA2B,CAAC,OAAO,EAAE,iBAAiB,GAAG,gBAAgB,GAAG,SAAS,GAAG,OAAO,CAAC,gBAAgB,GAAG,SAAS,CAAC,CAAA;QAC7H,2EAA2E;QAC3E,sBAAsB,CAAC,KAAK,EAAE,KAAK,GAAG,IAAI,CAAA;KAC3C;CACF;AAED,cAAc,YAAY,CAAA;AAC1B,OAAO,EAAE,gBAAgB,EAAE,WAAW,EAAE,iBAAiB,EAAE,KAAK,eAAe,EAAE,MAAM,aAAa,CAAA;AACpG,OAAO,EAAE,gBAAgB,EAAE,MAAM,iBAAiB,CAAA;AAClD,OAAO,EAAE,oBAAoB,EAAE,KAAK,aAAa,EAAE,MAAM,YAAY,CAAA;AAErE,wDAAwD;AACxD,eAAO,MAAM,IAAI,mBAAmB,CAAA;AAEpC,sFAAsF;AACtF,eAAO,MAAM,MAAM,UAAwB,CAAA;AAE3C;;;;;;GAMG;AACH,eAAO,MAAM,MAAM;;;;;;;;;;;;;;;;aAQjB,CAAA;AAEF,yGAAyG;AACzG,MAAM,WAAW,WAAW;IAC1B,wCAAwC;IACxC,QAAQ,CAAC,UAAU,EAAE,gBAAgB,CAAA;IACrC,mFAAmF;IACnF,QAAQ,CAAC,OAAO,EAAE,MAAM,eAAe,GAAG,SAAS,CAAA;IACnD,gCAAgC;IAChC,QAAQ,CAAC,GAAG,EAAE,eAAe,CAAA;IAC7B,gCAAgC;IAChC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,2EAA2E;IAC3E,QAAQ,CAAC,QAAQ,EAAE,MAAM,IAAI,CAAA;IAC7B,kDAAkD;IAClD,QAAQ,CAAC,cAAc,EAAE,CAAC,KAAK,EAAE,KAAK,KAAK,IAAI,CAAA;CAChD;AAED;;;;GAIG;AACH,wBAAgB,cAAc,CAAC,IAAI,EAAE,WAAW,GAAG,uBAAuB,CAkBzE;AAED;;;;GAIG;AACH,wBAAgB,KAAK,CAAC,GAAG,EAAE,OAAO,EAAE,MAAM,EAAE,mBAAmB,GAAG,IAAI,CA0DrE"}
|
package/lib/index.js
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The realtime agent consumer: answers what the voice model delegates.
|
|
3
|
+
*
|
|
4
|
+
* A **function plugin** on the `dsh-realtime` seam. It holds a live session, records the conversation,
|
|
5
|
+
* and when the model raises a delegation it asks the application — over the event bus, with `serial`
|
|
6
|
+
* semantics so the first responder to answer wins — and delivers the result, or says plainly that it
|
|
7
|
+
* cannot be handled.
|
|
8
|
+
*
|
|
9
|
+
* Per the harness convention it named-exports `name` / `inject` / `Config` / `apply` and has **no
|
|
10
|
+
* default export** — adding one makes the Loader discard this plugin's namespace, so the plugin would
|
|
11
|
+
* load and contribute nothing.
|
|
12
|
+
*
|
|
13
|
+
* An application answers by listening:
|
|
14
|
+
*
|
|
15
|
+
* ```ts
|
|
16
|
+
* ctx.on('realtime-agent/delegation', (request) => {
|
|
17
|
+
* return { text: lookItUp(request.transcript), mode: 'spoken' }
|
|
18
|
+
* })
|
|
19
|
+
* ```
|
|
20
|
+
*
|
|
21
|
+
* @module dsh-realtime-agent
|
|
22
|
+
*/
|
|
23
|
+
import Schema from '@deepseek-ai/schemastery';
|
|
24
|
+
import { answerDelegation } from './bridge.js';
|
|
25
|
+
import { voiceToolDefinitions } from './tools.js';
|
|
26
|
+
import { TranscriptBuffer } from './transcript.js';
|
|
27
|
+
export * from './types.js';
|
|
28
|
+
export { answerDelegation, boundAppend, UNANSWERED_NOTICE } from './bridge.js';
|
|
29
|
+
export { TranscriptBuffer } from './transcript.js';
|
|
30
|
+
export { voiceToolDefinitions } from './tools.js';
|
|
31
|
+
/** Plugin name, as it appears in Loader diagnostics. */
|
|
32
|
+
export const name = 'realtime-agent';
|
|
33
|
+
/** This plugin drives the realtime seam and publishes tools, so it waits for both. */
|
|
34
|
+
export const inject = ['realtime', 'tools'];
|
|
35
|
+
/**
|
|
36
|
+
* Validated configuration.
|
|
37
|
+
*
|
|
38
|
+
* `autoStart` defaults to `false`: mounting this plugin must not, by itself, open a socket or spend
|
|
39
|
+
* credit. `voice` and `instructions` are omitted unless set, so the provider default applies rather
|
|
40
|
+
* than an empty string the provider would reject.
|
|
41
|
+
*/
|
|
42
|
+
export const Config = Schema.object({
|
|
43
|
+
provider: Schema.string().default('openai-live').description('Registered realtime route to open a session on'),
|
|
44
|
+
model: Schema.string().default('gpt-live-1').description('Voice model to request'),
|
|
45
|
+
voice: Schema.string().required(false).description('Output voice; omitted leaves the provider default'),
|
|
46
|
+
instructions: Schema.string().required(false).description('Opening instructions for the live session'),
|
|
47
|
+
autoStart: Schema.boolean().default(false).description('Open a session as soon as the plugin mounts'),
|
|
48
|
+
delegationTimeoutMs: Schema.number().default(10_000).description('Bound on waiting for a responder'),
|
|
49
|
+
maxTranscriptChars: Schema.number().default(6_000).description('Character budget for the delegation transcript'),
|
|
50
|
+
});
|
|
51
|
+
/**
|
|
52
|
+
* Build the session handlers.
|
|
53
|
+
* @param deps - the session, the transcript and the asker.
|
|
54
|
+
* @returns handlers the adapter can invoke for the life of the session.
|
|
55
|
+
*/
|
|
56
|
+
export function createHandlers(deps) {
|
|
57
|
+
return {
|
|
58
|
+
onTranscript: (fragment) => {
|
|
59
|
+
deps.transcript.append(fragment.kind, fragment.text, fragment.final);
|
|
60
|
+
},
|
|
61
|
+
onDelegation: (delegation) => {
|
|
62
|
+
const session = deps.session();
|
|
63
|
+
// A delegation cannot be answered into a session that is gone. Speaking into a closed transport
|
|
64
|
+
// would be a failure raised from inside a handler, which is the worst place to raise one.
|
|
65
|
+
if (session === undefined)
|
|
66
|
+
return;
|
|
67
|
+
// Handlers are synchronous, so the answer is dispatched rather than awaited. A rejection is
|
|
68
|
+
// reported rather than thrown: an unanswered delegation is already the failure path.
|
|
69
|
+
void answerDelegation(session, delegation, deps.transcript.lines(), deps.ask, deps.timeoutMs)
|
|
70
|
+
.catch(deps.onSessionError);
|
|
71
|
+
},
|
|
72
|
+
onClosed: () => { deps.onClosed(); },
|
|
73
|
+
onError: (error) => { deps.onSessionError(error); },
|
|
74
|
+
};
|
|
75
|
+
}
|
|
76
|
+
/**
|
|
77
|
+
* Hold a session and bridge what the voice model delegates.
|
|
78
|
+
* @param ctx - the Cordis context, which must already provide the `realtime` service.
|
|
79
|
+
* @param config - validated configuration.
|
|
80
|
+
*/
|
|
81
|
+
export function apply(ctx, config) {
|
|
82
|
+
const transcript = new TranscriptBuffer(config.maxTranscriptChars);
|
|
83
|
+
let session;
|
|
84
|
+
const handlers = createHandlers({
|
|
85
|
+
transcript,
|
|
86
|
+
session: () => session,
|
|
87
|
+
ask: request => ctx.serial('realtime-agent/delegation', request),
|
|
88
|
+
timeoutMs: config.delegationTimeoutMs,
|
|
89
|
+
onClosed: () => { session = undefined; },
|
|
90
|
+
onSessionError: (error) => { ctx.emit('realtime-agent/error', error); },
|
|
91
|
+
});
|
|
92
|
+
/**
|
|
93
|
+
* Open a session, or return the one already open.
|
|
94
|
+
*
|
|
95
|
+
* Idempotent on purpose: `voice_start` is safe to call repeatedly, and a model that calls it twice
|
|
96
|
+
* must not end up with two live sessions it cannot see.
|
|
97
|
+
*/
|
|
98
|
+
const open = async () => {
|
|
99
|
+
const current = session;
|
|
100
|
+
if (current !== undefined)
|
|
101
|
+
return current;
|
|
102
|
+
const opened = await ctx.realtime.session({
|
|
103
|
+
provider: config.provider,
|
|
104
|
+
model: config.model,
|
|
105
|
+
...config.voice === undefined ? {} : { voice: config.voice },
|
|
106
|
+
...config.instructions === undefined ? {} : { instructions: config.instructions },
|
|
107
|
+
handlers,
|
|
108
|
+
});
|
|
109
|
+
session = opened;
|
|
110
|
+
return opened;
|
|
111
|
+
};
|
|
112
|
+
/**
|
|
113
|
+
* Close the session, dropping the reference first.
|
|
114
|
+
*
|
|
115
|
+
* Dropping it before the close settles means a delegation arriving during teardown is ignored rather
|
|
116
|
+
* than answered into a transport that is going away.
|
|
117
|
+
*/
|
|
118
|
+
const stop = async () => {
|
|
119
|
+
const current = session;
|
|
120
|
+
session = undefined;
|
|
121
|
+
await current?.close();
|
|
122
|
+
};
|
|
123
|
+
// Tools are an effect, like every other contribution this plugin makes: the fiber that mounted them
|
|
124
|
+
// releases them, so there is no separate teardown path to forget.
|
|
125
|
+
ctx.effect(function* () {
|
|
126
|
+
const disposers = voiceToolDefinitions({ session: () => session, start: open, stop })
|
|
127
|
+
.map(definition => ctx.tools.register(definition));
|
|
128
|
+
yield () => { for (const dispose of disposers)
|
|
129
|
+
dispose(); };
|
|
130
|
+
}, 'realtime-agent.tools');
|
|
131
|
+
if (config.autoStart) {
|
|
132
|
+
// `apply` is synchronous, so a failed open cannot be thrown from it — it is reported on the bus
|
|
133
|
+
// instead. The plugin stays valid and a later start may succeed.
|
|
134
|
+
void open().catch((error) => { ctx.emit('realtime-agent/error', error); });
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
//# sourceMappingURL=index.js.map
|
package/lib/index.js.map
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;GAqBG;AAEH,OAAO,MAAM,MAAM,0BAA0B,CAAA;AAG7C,OAAO,EAAE,gBAAgB,EAAwB,MAAM,aAAa,CAAA;AACpE,OAAO,EAAE,oBAAoB,EAAE,MAAM,YAAY,CAAA;AACjD,OAAO,EAAE,gBAAgB,EAAE,MAAM,iBAAiB,CAAA;AAkBlD,cAAc,YAAY,CAAA;AAC1B,OAAO,EAAE,gBAAgB,EAAE,WAAW,EAAE,iBAAiB,EAAwB,MAAM,aAAa,CAAA;AACpG,OAAO,EAAE,gBAAgB,EAAE,MAAM,iBAAiB,CAAA;AAClD,OAAO,EAAE,oBAAoB,EAAsB,MAAM,YAAY,CAAA;AAErE,wDAAwD;AACxD,MAAM,CAAC,MAAM,IAAI,GAAG,gBAAgB,CAAA;AAEpC,sFAAsF;AACtF,MAAM,CAAC,MAAM,MAAM,GAAG,CAAC,UAAU,EAAE,OAAO,CAAC,CAAA;AAE3C;;;;;;GAMG;AACH,MAAM,CAAC,MAAM,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC;IAClC,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,aAAa,CAAC,CAAC,WAAW,CAAC,gDAAgD,CAAC;IAC9G,KAAK,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,YAAY,CAAC,CAAC,WAAW,CAAC,wBAAwB,CAAC;IAClF,KAAK,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC,WAAW,CAAC,mDAAmD,CAAC;IACvG,YAAY,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC,WAAW,CAAC,2CAA2C,CAAC;IACtG,SAAS,EAAE,MAAM,CAAC,OAAO,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,WAAW,CAAC,6CAA6C,CAAC;IACrG,mBAAmB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,WAAW,CAAC,kCAAkC,CAAC;IACpG,kBAAkB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,WAAW,CAAC,gDAAgD,CAAC;CACjH,CAAC,CAAA;AAkBF;;;;GAIG;AACH,MAAM,UAAU,cAAc,CAAC,IAAiB;IAC9C,OAAO;QACL,YAAY,EAAE,CAAC,QAA4B,EAAQ,EAAE;YACnD,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,EAAE,QAAQ,CAAC,IAAI,EAAE,QAAQ,CAAC,KAAK,CAAC,CAAA;QACtE,CAAC;QACD,YAAY,EAAE,CAAC,UAA8B,EAAQ,EAAE;YACrD,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,EAAE,CAAA;YAC9B,gGAAgG;YAChG,0FAA0F;YAC1F,IAAI,OAAO,KAAK,SAAS;gBAAE,OAAM;YACjC,4FAA4F;YAC5F,qFAAqF;YACrF,KAAK,gBAAgB,CAAC,OAAO,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,CAAC,KAAK,EAAE,EAAE,IAAI,CAAC,GAAG,EAAE,IAAI,CAAC,SAAS,CAAC;iBAC1F,KAAK,CAAC,IAAI,CAAC,cAAc,CAAC,CAAA;QAC/B,CAAC;QACD,QAAQ,EAAE,GAAS,EAAE,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAA,CAAC,CAAC;QACzC,OAAO,EAAE,CAAC,KAAY,EAAQ,EAAE,GAAG,IAAI,CAAC,cAAc,CAAC,KAAK,CAAC,CAAA,CAAC,CAAC;KAChE,CAAA;AACH,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,KAAK,CAAC,GAAY,EAAE,MAA2B;IAC7D,MAAM,UAAU,GAAG,IAAI,gBAAgB,CAAC,MAAM,CAAC,kBAAkB,CAAC,CAAA;IAClE,IAAI,OAAoC,CAAA;IAExC,MAAM,QAAQ,GAAG,cAAc,CAAC;QAC9B,UAAU;QACV,OAAO,EAAE,GAAG,EAAE,CAAC,OAAO;QACtB,GAAG,EAAE,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,MAAM,CAAC,2BAA2B,EAAE,OAAO,CAAC;QAChE,SAAS,EAAE,MAAM,CAAC,mBAAmB;QACrC,QAAQ,EAAE,GAAG,EAAE,GAAG,OAAO,GAAG,SAAS,CAAA,CAAC,CAAC;QACvC,cAAc,EAAE,CAAC,KAAK,EAAE,EAAE,GAAG,GAAG,CAAC,IAAI,CAAC,sBAAsB,EAAE,KAAK,CAAC,CAAA,CAAC,CAAC;KACvE,CAAC,CAAA;IAEF;;;;;OAKG;IACH,MAAM,IAAI,GAAG,KAAK,IAA8B,EAAE;QAChD,MAAM,OAAO,GAAG,OAAO,CAAA;QACvB,IAAI,OAAO,KAAK,SAAS;YAAE,OAAO,OAAO,CAAA;QACzC,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC;YACxC,QAAQ,EAAE,MAAM,CAAC,QAAQ;YACzB,KAAK,EAAE,MAAM,CAAC,KAAK;YACnB,GAAG,MAAM,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE;YAC5D,GAAG,MAAM,CAAC,YAAY,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,YAAY,EAAE,MAAM,CAAC,YAAY,EAAE;YACjF,QAAQ;SACT,CAAC,CAAA;QACF,OAAO,GAAG,MAAM,CAAA;QAChB,OAAO,MAAM,CAAA;IACf,CAAC,CAAA;IAED;;;;;OAKG;IACH,MAAM,IAAI,GAAG,KAAK,IAAmB,EAAE;QACrC,MAAM,OAAO,GAAG,OAAO,CAAA;QACvB,OAAO,GAAG,SAAS,CAAA;QACnB,MAAM,OAAO,EAAE,KAAK,EAAE,CAAA;IACxB,CAAC,CAAA;IAED,oGAAoG;IACpG,kEAAkE;IAClE,GAAG,CAAC,MAAM,CAAC,QAAQ,CAAC;QAClB,MAAM,SAAS,GAAG,oBAAoB,CAAC,EAAE,OAAO,EAAE,GAAG,EAAE,CAAC,OAAO,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;aAClF,GAAG,CAAC,UAAU,CAAC,EAAE,CAAC,GAAG,CAAC,KAAK,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAA;QACpD,MAAM,GAAG,EAAE,GAAG,KAAK,MAAM,OAAO,IAAI,SAAS;YAAE,OAAO,EAAE,CAAA,CAAC,CAAC,CAAA;IAC5D,CAAC,EAAE,sBAAsB,CAAC,CAAA;IAE1B,IAAI,MAAM,CAAC,SAAS,EAAE,CAAC;QACrB,gGAAgG;QAChG,iEAAiE;QACjE,KAAK,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,KAAY,EAAE,EAAE,GAAG,GAAG,CAAC,IAAI,CAAC,sBAAsB,EAAE,KAAK,CAAC,CAAA,CAAC,CAAC,CAAC,CAAA;IAClF,CAAC;AACH,CAAC"}
|
package/lib/tools.d.ts
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
import { type ToolDefinition } from '@deepseek-ai/dsh-tools';
|
|
2
|
+
import type { RealtimeSession } from 'dsh-realtime';
|
|
3
|
+
/** The session controls the voice tools act on. */
|
|
4
|
+
export interface VoiceToolDeps {
|
|
5
|
+
/** The live session, or `undefined` when none is open. */
|
|
6
|
+
readonly session: () => RealtimeSession | undefined;
|
|
7
|
+
/** Open a session, or return the one already open. */
|
|
8
|
+
readonly start: () => Promise<RealtimeSession>;
|
|
9
|
+
/** Close the open session, if any. */
|
|
10
|
+
readonly stop: () => Promise<void>;
|
|
11
|
+
}
|
|
12
|
+
/**
|
|
13
|
+
* The tools that let an agent drive a voice session.
|
|
14
|
+
*
|
|
15
|
+
* Deliberately a small surface with one concern — driving the session — rather than a chat UI in
|
|
16
|
+
* tool form. Reading what was said needs no tool: a delegation already arrives carrying the
|
|
17
|
+
* conversation, which is the whole reason the consumer exists.
|
|
18
|
+
* @param deps - the session controls.
|
|
19
|
+
* @returns definitions ready to register on the tools service.
|
|
20
|
+
*/
|
|
21
|
+
export declare function voiceToolDefinitions(deps: VoiceToolDeps): ToolDefinition[];
|
|
22
|
+
//# sourceMappingURL=tools.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"tools.d.ts","sourceRoot":"","sources":["../src/tools.ts"],"names":[],"mappings":"AAAA,OAAO,EAA+B,KAAK,cAAc,EAAwB,MAAM,wBAAwB,CAAA;AAC/G,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,cAAc,CAAA;AAqBnD,mDAAmD;AACnD,MAAM,WAAW,aAAa;IAC5B,0DAA0D;IAC1D,QAAQ,CAAC,OAAO,EAAE,MAAM,eAAe,GAAG,SAAS,CAAA;IACnD,sDAAsD;IACtD,QAAQ,CAAC,KAAK,EAAE,MAAM,OAAO,CAAC,eAAe,CAAC,CAAA;IAC9C,sCAAsC;IACtC,QAAQ,CAAC,IAAI,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAA;CACnC;AAuBD;;;;;;;;GAQG;AACH,wBAAgB,oBAAoB,CAAC,IAAI,EAAE,aAAa,GAAG,cAAc,EAAE,CA6C1E"}
|
package/lib/tools.js
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
import { defineTool } from '@deepseek-ai/dsh-tools';
|
|
2
|
+
/**
|
|
3
|
+
* A canonical output declaration rendered as JSON.
|
|
4
|
+
*
|
|
5
|
+
* `defineTool` requires an `output` with a schema **and** a `render` projection, and the harness ships
|
|
6
|
+
* no helper for the common case — where the value is for the model to read rather than for a bespoke
|
|
7
|
+
* presenter. Six lines here rather than a dependency on another plugin's private helper.
|
|
8
|
+
* @param schema - the canonical value's author-facing schema.
|
|
9
|
+
* @returns the output declaration the tool definition wants.
|
|
10
|
+
*/
|
|
11
|
+
function jsonOutput(schema) {
|
|
12
|
+
return {
|
|
13
|
+
schema,
|
|
14
|
+
render: (_args, value) => [{ type: 'text', text: JSON.stringify(value) }],
|
|
15
|
+
};
|
|
16
|
+
}
|
|
17
|
+
const STARTED_VALUE = {
|
|
18
|
+
type: 'object',
|
|
19
|
+
additionalProperties: false,
|
|
20
|
+
properties: { sessionId: { type: 'string', required: true } },
|
|
21
|
+
};
|
|
22
|
+
const STOPPED_VALUE = {
|
|
23
|
+
type: 'object',
|
|
24
|
+
additionalProperties: false,
|
|
25
|
+
properties: { closed: { type: 'boolean', required: true } },
|
|
26
|
+
};
|
|
27
|
+
const SAID_VALUE = {
|
|
28
|
+
type: 'object',
|
|
29
|
+
additionalProperties: false,
|
|
30
|
+
properties: {
|
|
31
|
+
spoken: { type: 'boolean', required: true },
|
|
32
|
+
characters: { type: 'number', required: true },
|
|
33
|
+
},
|
|
34
|
+
};
|
|
35
|
+
/**
|
|
36
|
+
* The tools that let an agent drive a voice session.
|
|
37
|
+
*
|
|
38
|
+
* Deliberately a small surface with one concern — driving the session — rather than a chat UI in
|
|
39
|
+
* tool form. Reading what was said needs no tool: a delegation already arrives carrying the
|
|
40
|
+
* conversation, which is the whole reason the consumer exists.
|
|
41
|
+
* @param deps - the session controls.
|
|
42
|
+
* @returns definitions ready to register on the tools service.
|
|
43
|
+
*/
|
|
44
|
+
export function voiceToolDefinitions(deps) {
|
|
45
|
+
return [
|
|
46
|
+
defineTool({
|
|
47
|
+
name: 'voice_start',
|
|
48
|
+
description: 'Open the live voice session so you can be heard. Safe to call when one is already open.',
|
|
49
|
+
parameters: {},
|
|
50
|
+
output: jsonOutput(STARTED_VALUE),
|
|
51
|
+
async execute() {
|
|
52
|
+
const session = await deps.start();
|
|
53
|
+
return { sessionId: session.id };
|
|
54
|
+
},
|
|
55
|
+
}),
|
|
56
|
+
defineTool({
|
|
57
|
+
name: 'voice_stop',
|
|
58
|
+
description: 'End the live voice session. Safe to call when none is open.',
|
|
59
|
+
parameters: {},
|
|
60
|
+
output: jsonOutput(STOPPED_VALUE),
|
|
61
|
+
async execute() {
|
|
62
|
+
// Reported rather than thrown: stopping a session that is not running has achieved what the
|
|
63
|
+
// caller asked for, so it is a success that says it had nothing to do.
|
|
64
|
+
const wasOpen = deps.session() !== undefined;
|
|
65
|
+
await deps.stop();
|
|
66
|
+
return { closed: wasOpen };
|
|
67
|
+
},
|
|
68
|
+
}),
|
|
69
|
+
defineTool({
|
|
70
|
+
name: 'voice_say',
|
|
71
|
+
description: 'Say something out loud in the live voice session, in your own voice.',
|
|
72
|
+
parameters: { text: { type: 'string', required: true, description: 'What to say aloud.' } },
|
|
73
|
+
output: jsonOutput(SAID_VALUE),
|
|
74
|
+
async execute(args) {
|
|
75
|
+
const session = deps.session();
|
|
76
|
+
// Thrown, not returned as a value. A tool-thrown failure is the registry's own failure
|
|
77
|
+
// channel, and reporting this as a successful result would leave the model believing it had
|
|
78
|
+
// spoken when nothing was said.
|
|
79
|
+
if (session === undefined) {
|
|
80
|
+
throw new Error('no voice session is open — call voice_start first');
|
|
81
|
+
}
|
|
82
|
+
await session.appendCommentary(args.text);
|
|
83
|
+
return { spoken: true, characters: args.text.length };
|
|
84
|
+
},
|
|
85
|
+
}),
|
|
86
|
+
];
|
|
87
|
+
}
|
|
88
|
+
//# sourceMappingURL=tools.js.map
|
package/lib/tools.js.map
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"tools.js","sourceRoot":"","sources":["../src/tools.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAA8D,MAAM,wBAAwB,CAAA;AAG/G;;;;;;;;GAQG;AACH,SAAS,UAAU,CAAkC,MAAS;IAI5D,OAAO;QACL,MAAM;QACN,MAAM,EAAE,CAAC,KAAc,EAAE,KAAoB,EAAE,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,EAAE,CAAC;KAClG,CAAA;AACH,CAAC;AAYD,MAAM,aAAa,GAAG;IACpB,IAAI,EAAE,QAAQ;IACd,oBAAoB,EAAE,KAAK;IAC3B,UAAU,EAAE,EAAE,SAAS,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;CACrD,CAAA;AAEV,MAAM,aAAa,GAAG;IACpB,IAAI,EAAE,QAAQ;IACd,oBAAoB,EAAE,KAAK;IAC3B,UAAU,EAAE,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;CACnD,CAAA;AAEV,MAAM,UAAU,GAAG;IACjB,IAAI,EAAE,QAAQ;IACd,oBAAoB,EAAE,KAAK;IAC3B,UAAU,EAAE;QACV,MAAM,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,QAAQ,EAAE,IAAI,EAAE;QAC3C,UAAU,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,QAAQ,EAAE,IAAI,EAAE;KAC/C;CACO,CAAA;AAEV;;;;;;;;GAQG;AACH,MAAM,UAAU,oBAAoB,CAAC,IAAmB;IACtD,OAAO;QACL,UAAU,CAAC;YACT,IAAI,EAAE,aAAa;YACnB,WAAW,EAAE,yFAAyF;YACtG,UAAU,EAAE,EAAE;YACd,MAAM,EAAE,UAAU,CAAC,aAAa,CAAC;YACjC,KAAK,CAAC,OAAO;gBACX,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,KAAK,EAAE,CAAA;gBAClC,OAAO,EAAE,SAAS,EAAE,OAAO,CAAC,EAAE,EAAE,CAAA;YAClC,CAAC;SACF,CAAC;QAEF,UAAU,CAAC;YACT,IAAI,EAAE,YAAY;YAClB,WAAW,EAAE,6DAA6D;YAC1E,UAAU,EAAE,EAAE;YACd,MAAM,EAAE,UAAU,CAAC,aAAa,CAAC;YACjC,KAAK,CAAC,OAAO;gBACX,4FAA4F;gBAC5F,uEAAuE;gBACvE,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,EAAE,KAAK,SAAS,CAAA;gBAC5C,MAAM,IAAI,CAAC,IAAI,EAAE,CAAA;gBACjB,OAAO,EAAE,MAAM,EAAE,OAAO,EAAE,CAAA;YAC5B,CAAC;SACF,CAAC;QAEF,UAAU,CAAC;YACT,IAAI,EAAE,WAAW;YACjB,WAAW,EAAE,sEAAsE;YACnF,UAAU,EAAE,EAAE,IAAI,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,QAAQ,EAAE,IAAI,EAAE,WAAW,EAAE,oBAAoB,EAAE,EAAE;YAC3F,MAAM,EAAE,UAAU,CAAC,UAAU,CAAC;YAC9B,KAAK,CAAC,OAAO,CAAC,IAAI;gBAChB,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,EAAE,CAAA;gBAC9B,uFAAuF;gBACvF,4FAA4F;gBAC5F,gCAAgC;gBAChC,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;oBAC1B,MAAM,IAAI,KAAK,CAAC,mDAAmD,CAAC,CAAA;gBACtE,CAAC;gBACD,MAAM,OAAO,CAAC,gBAAgB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;gBACzC,OAAO,EAAE,MAAM,EAAE,IAAI,EAAE,UAAU,EAAE,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,CAAA;YACvD,CAAC;SACF,CAAC;KACH,CAAA;AACH,CAAC"}
|