@anam-ai/js-sdk 1.4.0-alpha.3 → 1.4.0-rc.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +51 -9
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -88,7 +88,7 @@ Regardless of whether you initialise the client using an API key or session toke
|
|
|
88
88
|
|
|
89
89
|
[See here](#starting-a-session-in-production-environments) for an example sequence diagram of starting a session in production environments.
|
|
90
90
|
|
|
91
|
-
## Using the
|
|
91
|
+
## Using the Talk command to control persona output
|
|
92
92
|
|
|
93
93
|
Sometimes during a persona session you may wish to force a response from the persona. For example when the user interacts with an element on the page or when you have disabled the Anam LLM in order to use your own. To do this you can use the `talk` method of the Anam client.
|
|
94
94
|
|
|
@@ -96,6 +96,47 @@ Sometimes during a persona session you may wish to force a response from the per
|
|
|
96
96
|
anamClient.talk('Content to say');
|
|
97
97
|
```
|
|
98
98
|
|
|
99
|
+
The `talk` method will send a `talk` command to the persona which will respond by speaking the provided content.
|
|
100
|
+
|
|
101
|
+
## Streaming Talk input
|
|
102
|
+
|
|
103
|
+
You may want to stream a particular message to the persona in multiple chunks. For example when you are streaming output from a custom LLM and want to reduce latency by sending the message in chunks. To do this you can use the `createTalkMessageStream` method to create a `TalkMessageStream` instance and use the `streamMessageChunk` method to send the message in chunks.
|
|
104
|
+
|
|
105
|
+
This approach can be useful to reduce latency when streaming output from a custom LLM, however it does introduce some complexity due to the need to handle interrupts and end of speech.
|
|
106
|
+
|
|
107
|
+
Example usage:
|
|
108
|
+
|
|
109
|
+
```typescript
|
|
110
|
+
const talkMessageStream = anamClient.createTalkMessageStream();
|
|
111
|
+
const chunks = ['He', 'l', 'lo', ', how are you?'];
|
|
112
|
+
|
|
113
|
+
for (const chunk of chunks) {
|
|
114
|
+
if (talkMessageStream.isActive()) {
|
|
115
|
+
talkMessageStream.streamMessageChunk(
|
|
116
|
+
chunk,
|
|
117
|
+
chunk === chunks[chunks.length - 1],
|
|
118
|
+
);
|
|
119
|
+
}
|
|
120
|
+
}
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
If a sentence is not interrupted, you must signal the end of the speech yourself by calling `streamMessageChunk` with the `endOfSpeech` parameter set to `true` or by calling `talkMessageStream.endMessage()`. If a talkMessageStream is already closed, either due to an interrupt or end of speech, you do not need to signal end of speech.
|
|
124
|
+
|
|
125
|
+
**Important note**: One talkMessageStream represents one "turn" in the conversation. Once that turn is over, the object can no longer be used and you must create a new `TalkMessageStream` using `streamMessageChunk`.
|
|
126
|
+
|
|
127
|
+
There are two ways a "turn" can end and a `TalkMessageStream` closed:
|
|
128
|
+
|
|
129
|
+
1. End of speech: `streamMessageChunk` is called with the `endOfSpeech` parameter set to `true` or `endMessage` is called.
|
|
130
|
+
2. Interrupted by user speech: The user speaks during the stream causing an `AnamEvent.TALK_STREAM_INTERRUPTED` event to fire and the `TalkMessageStream` object to be closed automatically.
|
|
131
|
+
|
|
132
|
+
In both of these cases the `TalkMessageStream` object is closed and no longer usable. If you try to call `streamMessageChunk` or `endMessage` on a closed `TalkMessageStream` object you will be met with an error. To handle this you can check the `isActive()` method of the `TalkMessageStream` object or listen for the `AnamEvent.TALK_STREAM_INTERRUPTED` event.
|
|
133
|
+
|
|
134
|
+
#### Correlation IDs
|
|
135
|
+
|
|
136
|
+
The `streamMessageChunk` method accepts an optional `correlationId` parameter. **This should be unique for every `TalkMessageStream` instance.** When a talk stream is interrupted by user speech the interrupted stream's `correlationId` will be sent in the `AnamEvent.TALK_STREAM_INTERRUPTED` event.
|
|
137
|
+
|
|
138
|
+
Currently only one `TalkMessageStream` can be active at a time, so this is not strictly necessary.
|
|
139
|
+
|
|
99
140
|
## Controlling the input audio
|
|
100
141
|
|
|
101
142
|
### Audio input state
|
|
@@ -232,14 +273,15 @@ anamClient.addListener(AnamEvent.MESSAGE_HISTORY_UPDATED, (messages) => {
|
|
|
232
273
|
|
|
233
274
|
### Event Types
|
|
234
275
|
|
|
235
|
-
| Event Name | Description
|
|
236
|
-
| ------------------------------- |
|
|
237
|
-
| `CONNECTION_ESTABLISHED` | Called when the direct connection between the browser and the Anam Engine has been established.
|
|
238
|
-
| `CONNECTION_CLOSED` | Called when the direct connection between the browser and the Anam Engine has been closed.
|
|
239
|
-
| `VIDEO_PLAY_STARTED` | When streaming directly to a video element this event is fired when the first frames start playing. Useful for removing any loading indicators during connection.
|
|
240
|
-
| `MESSAGE_HISTORY_UPDATED` | Called with the message history transcription of the current session each time the user or the persona finishes speaking.
|
|
241
|
-
| `MESSAGE_STREAM_EVENT_RECEIVED` | For persona speech, this stream is updated with each transcribed speech chunk as the persona is speaking. For the user speech this stream is updated with a complete transcription of the user's sentence once they finish speaking
|
|
242
|
-
| `INPUT_AUDIO_STREAM_STARTED` | Called with the users input audio stream when microphone input has been initialised.
|
|
276
|
+
| Event Name | Description |
|
|
277
|
+
| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
278
|
+
| `CONNECTION_ESTABLISHED` | Called when the direct connection between the browser and the Anam Engine has been established. |
|
|
279
|
+
| `CONNECTION_CLOSED` | Called when the direct connection between the browser and the Anam Engine has been closed. |
|
|
280
|
+
| `VIDEO_PLAY_STARTED` | When streaming directly to a video element this event is fired when the first frames start playing. Useful for removing any loading indicators during connection. |
|
|
281
|
+
| `MESSAGE_HISTORY_UPDATED` | Called with the message history transcription of the current session each time the user or the persona finishes speaking. |
|
|
282
|
+
| `MESSAGE_STREAM_EVENT_RECEIVED` | For persona speech, this stream is updated with each transcribed speech chunk as the persona is speaking. For the user speech this stream is updated with a complete transcription of the user's sentence once they finish speaking |
|
|
283
|
+
| `INPUT_AUDIO_STREAM_STARTED` | Called with the users input audio stream when microphone input has been initialised. |
|
|
284
|
+
| `TALK_STREAM_INTERRUPTED` | Called when the user interrupts the current `TalkMessageStream` by speaking. The interrupted stream's `correlationId` will be sent in the event. The TalkMessageStream object is automatically closed by the SDK but this event is provided to allow for any additional actions. |
|
|
243
285
|
|
|
244
286
|
# Personas
|
|
245
287
|
|