@d-id/client-sdk 3.0.0-staging.416 → 3.0.0-staging.417
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +43 -116
- package/dist/{index-DCAt0P7O.js → index-BhhDKHW-.js} +6 -4
- package/dist/index-BhhDKHW-.js.map +1 -0
- package/dist/index.d.ts +318 -102
- package/dist/index.js +2 -1
- package/dist/index.js.map +1 -0
- package/dist/index.umd.cjs +2 -1
- package/dist/livekit-client.esm-QkbCKsng.js +1 -0
- package/dist/{livekit-manager-Cj0YXe2U.js → livekit-manager-DF70NOGW.js} +2 -1
- package/dist/livekit-manager-DF70NOGW.js.map +1 -0
- package/package.json +4 -2
package/README.md
CHANGED
|
@@ -1,148 +1,75 @@
|
|
|
1
1
|
# D-ID Client SDK
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://www.npmjs.com/package/@d-id/client-sdk)
|
|
4
|
+
[](https://github.com/de-id/agents-sdk/blob/main/LICENSE)
|
|
5
|
+
[](https://sdk.d-id.com/)
|
|
4
6
|
|
|
5
|
-
|
|
6
|
-
<img style="width: auto; height: 200px; object-fit: contain;" src="https://create-images-results.d-id.com/api_docs/assets/agents_sdk_cover_v2.png" alt="Agents SDK Cover" />
|
|
7
|
-
<span style="width: 67%; text-align: justify;">
|
|
8
|
-
<br> This guide covers installing the SDK, connecting to an agent and where to find the full API reference.
|
|
9
|
-
</span>
|
|
10
|
-
</div>
|
|
7
|
+

|
|
11
8
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
The D-ID Agents SDK embeds your created Agents or real-time streaming avatars into web applications.
|
|
15
|
-
|
|
16
|
-
It wraps the D-ID Agents and Streams APIs.
|
|
17
|
-
|
|
18
|
-
The SDK supports three avatar types:
|
|
19
|
-
|
|
20
|
-
- **Talks (V2)** — Photo-based presenters using WebRTC streaming.
|
|
21
|
-
- **Clips (V3)** — Pre-built presenter avatars using WebRTC streaming.
|
|
22
|
-
- **Expressive (V4)** — Next-generation avatars using LiveKit-based streaming, supporting microphone input and always-on fluent mode.
|
|
9
|
+
The D-ID Agents SDK embeds an agent you built in D-ID Studio — or a real-time streaming avatar — into a web application. It wraps the D-ID Agents and Streams APIs behind one object: create an `AgentManager` for an agent, `connect()`, then `chat()` to have the agent answer with its own LLM or `speak()` to have it say exactly what you give it. The SDK handles the WebRTC or LiveKit session, the chat transcript and the reconnects.
|
|
23
10
|
|
|
24
11
|
**Please note:** This SDK is designed for front-end development only. The creation of Agents and Knowledge bases should be handled through the [Agents API](https://docs.d-id.com/docs/agent-quickstart) or directly within the [D-ID Studio](https://studio.d-id.com/agents).
|
|
25
12
|
|
|
26
|
-
>
|
|
27
|
-
|
|
28
|
-
## Getting Started
|
|
13
|
+
> **Version 3.0 is a breaking release.** Upgrading from 2.x? Read the [migration guide](https://sdk.d-id.com/documents/Migration_guide.html).
|
|
29
14
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
Follow these steps:
|
|
33
|
-
|
|
34
|
-
1. Log in to the [D-ID Studio](https://studio.d-id.com)
|
|
35
|
-
2. Create a new Agent with the required options - Image, voice, etc.
|
|
36
|
-
3. In the [Agents gallery](https://studio.d-id.com/agents), hover with your mouse over the created Agent, then click on the `[...]` button
|
|
37
|
-
4. Click on `</> Embed` button
|
|
38
|
-
5. Set the list of allowed domains for your Agent, for example: `http://localhost`
|
|
39
|
-
This is an additional security measurement: your Agent can be accessed only from the domains allowed by you.
|
|
40
|
-
6. In the code snippet section, fetch the `data-client-key` and the `data-agent-id`, these will be used later to access your Agent.
|
|
41
|
-
|
|
42
|
-
### Installation
|
|
43
|
-
|
|
44
|
-
In your front-end application folder, install the Agents SDK library using `npm`.
|
|
15
|
+
## Install
|
|
45
16
|
|
|
46
17
|
```shell
|
|
47
18
|
npm i @d-id/client-sdk
|
|
48
19
|
```
|
|
49
20
|
|
|
50
|
-
|
|
21
|
+
You need an agent id and a client key. Both come from the agent's `</> Embed` snippet in the [D-ID Studio agents gallery](https://studio.d-id.com/agents), which is also where you list the domains the agent may be used from — see [Getting started](https://sdk.d-id.com/documents/Getting_started.html).
|
|
51
22
|
|
|
52
|
-
|
|
23
|
+
## Quick start
|
|
53
24
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
1. Import the Agents SDK library
|
|
57
|
-
2. Paste the `data-agent-id` obtained in the prerequisites step in the `agentId` variable
|
|
58
|
-
3. Paste the `data-client-key` obtained in the prerequisites step in the `auth.clientKey` variable
|
|
59
|
-
4. Define an object called `callbacks`. `onSrcObjectReady` is mandatory: it hands you the media stream to render, so without it the agent has nowhere to play.
|
|
60
|
-
See [`AgentManagerCallbacks`](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html) in the API reference.
|
|
61
|
-
5. Define an object called `streamOptions` [optional — Talks (V2) and Clips (V3) agents only]
|
|
62
|
-
See [`StreamOptions`](https://sdk.d-id.com/interfaces/StreamOptions.html) in the API reference.
|
|
63
|
-
6. Create an instance of the `createAgentManager` object called `agentManager` with the values created above.
|
|
64
|
-
See [`AgentManager`](https://sdk.d-id.com/interfaces/AgentManager.html) in the API reference.
|
|
65
|
-
7. Open the session with `connect()`, and wait for it to resolve before speaking to the agent.
|
|
66
|
-
8. Make the agent talk: `speak()` says exactly what you give it, `chat()` has the agent answer with its own LLM.
|
|
67
|
-
|
|
68
|
-
Example:
|
|
69
|
-
|
|
70
|
-
```javascript
|
|
71
|
-
// 1. Import the Agents SDK library
|
|
25
|
+
```ts
|
|
72
26
|
import * as sdk from '@d-id/client-sdk';
|
|
73
27
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
// 4. Define the SDK callbacks functions in this object
|
|
81
|
-
const videoElement = document.getElementById('agent-video');
|
|
82
|
-
|
|
83
|
-
const callbacks = {
|
|
84
|
-
onSrcObjectReady(value) {
|
|
85
|
-
videoElement.srcObject = value;
|
|
86
|
-
},
|
|
87
|
-
onConnectionStateChange(state) {
|
|
88
|
-
console.log('connection:', state);
|
|
89
|
-
},
|
|
90
|
-
onNewMessage(messages, type) {
|
|
91
|
-
if (type === 'answer') {
|
|
92
|
-
console.log(messages[messages.length - 1].content);
|
|
93
|
-
}
|
|
28
|
+
const videoElement = document.getElementById('agent-video') as HTMLVideoElement;
|
|
29
|
+
const agentManager = await sdk.createAgentManager('agt_fumf1234', {
|
|
30
|
+
auth: { type: 'key', clientKey: 'YOUR_CLIENT_KEY' },
|
|
31
|
+
callbacks: {
|
|
32
|
+
onSrcObjectReady: stream => (videoElement.srcObject = stream),
|
|
33
|
+
onNewMessage: messages => console.log(messages[messages.length - 1].content),
|
|
94
34
|
},
|
|
95
|
-
};
|
|
35
|
+
});
|
|
96
36
|
|
|
97
|
-
// 5. Define the Stream Options object (Optional — Talks (V2) and Clips (V3) agents only)
|
|
98
|
-
let streamOptions = { compatibilityMode: 'auto', streamWarmup: true };
|
|
99
|
-
|
|
100
|
-
//....Rest of the APP's code here....//
|
|
101
|
-
//...................................//
|
|
102
|
-
|
|
103
|
-
// 6. Create the 'agentManager' instance with the values created above
|
|
104
|
-
let agentManager = await sdk.createAgentManager(agentId, { auth, callbacks, streamOptions });
|
|
105
|
-
|
|
106
|
-
// 7. Open the session
|
|
107
37
|
await agentManager.connect();
|
|
108
|
-
|
|
109
|
-
// 8. Make the agent talk
|
|
110
38
|
await agentManager.speak({ type: 'text', input: "Hi! I'm Alice!" });
|
|
111
39
|
await agentManager.chat('What is the distance to the moon?');
|
|
40
|
+
|
|
41
|
+
// Both resolve when the request is accepted, not when the agent has spoken.
|
|
42
|
+
window.addEventListener('beforeunload', () => void agentManager.disconnect());
|
|
112
43
|
```
|
|
113
44
|
|
|
114
|
-
##
|
|
45
|
+
## Next steps
|
|
46
|
+
|
|
47
|
+
The full API reference — every method, callback, option and type, generated from the source on each release — is at **[sdk.d-id.com](https://sdk.d-id.com/)**. Start with [`createAgentManager()`](https://sdk.d-id.com/functions/createAgentManager.html), [`AgentManager`](https://sdk.d-id.com/interfaces/AgentManager.html), [`AgentManagerCallbacks`](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html) and, on Talks (V2) and Clips (V3) agents, [`StreamOptions`](https://sdk.d-id.com/interfaces/StreamOptions.html).
|
|
48
|
+
|
|
49
|
+
The guides:
|
|
50
|
+
|
|
51
|
+
- [Getting started](https://sdk.d-id.com/documents/Getting_started.html) — credentials, the first connection, `chat()` and `speak()`, disconnecting.
|
|
52
|
+
- [Chat modes](https://sdk.d-id.com/documents/Chat_modes.html) — video, text-only or speak-only, and what each one creates on the wire.
|
|
53
|
+
- [Client tools](https://sdk.d-id.com/documents/Client_tools.html) — running your own functions when the agent's LLM calls them.
|
|
54
|
+
- [Expressive media](https://sdk.d-id.com/documents/Expressive_media.html) — microphone, camera, speech-to-text, interrupts and data-channel messages.
|
|
55
|
+
- [Handling errors](https://sdk.d-id.com/documents/Handling_errors.html) — which failures reject, which reach `onError`, and what to log.
|
|
56
|
+
- [Migration guide](https://sdk.d-id.com/documents/Migration_guide.html) — every breaking change from 2.x.
|
|
115
57
|
|
|
116
|
-
|
|
58
|
+
A working sample project in Vanilla JavaScript and Vite is in the [demo repository](https://github.com/de-id/Agents-SDK-Demo).
|
|
117
59
|
|
|
118
|
-
|
|
119
|
-
- [`AgentManager`](https://sdk.d-id.com/interfaces/AgentManager.html) — `connect()`, `speak()`, `chat()`, `interrupt()`, microphone and camera publishing, client tools
|
|
120
|
-
- [`AgentManagerCallbacks`](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html) — `onSrcObjectReady` (mandatory), `onVideoStateChange`, `onConnectionStateChange`, `onNewMessage`, …
|
|
121
|
-
- [`StreamOptions`](https://sdk.d-id.com/interfaces/StreamOptions.html) — Talks (V2) and Clips (V3) transport options
|
|
60
|
+
## Requirements
|
|
122
61
|
|
|
123
|
-
|
|
62
|
+
- **A modern browser with WebRTC.** The SDK streams the agent's video over WebRTC (Talks and Clips) or LiveKit (Expressive), and renders it into a `<video>` element you provide.
|
|
63
|
+
- **ESM and UMD builds.** `import` resolves to the ES module; `require` and a `<script>` tag resolve to the UMD bundle.
|
|
64
|
+
- **TypeScript types included.** They ship with the package; there is no separate `@types` to install.
|
|
65
|
+
- **Node is not a runtime target**, but importing the package on a server is safe: the module graph touches no browser globals at import time, so server-rendered frameworks (Next.js, Remix, SvelteKit, Astro) can evaluate it. Everything past `createAgentManager()` needs a browser.
|
|
124
66
|
|
|
125
|
-
|
|
126
|
-
This repository features a sample project crafted in Vanilla JavaScript and Vite, utilizing the Agents SDK to help you get started swiftly.
|
|
67
|
+
## Avatar types
|
|
127
68
|
|
|
128
|
-
|
|
69
|
+
- **Talks (V2)** — Photo-based presenters using WebRTC streaming.
|
|
70
|
+
- **Clips (V3)** — Pre-built presenter avatars using WebRTC streaming.
|
|
71
|
+
- **Expressive (V4)** — Next-generation avatars using LiveKit-based streaming, supporting microphone input and always-on fluent mode.
|
|
129
72
|
|
|
130
73
|
## Support
|
|
131
74
|
|
|
132
|
-
|
|
133
|
-
<img style="width: 37%; border-radius: 5px; object-fit: cover;" src="https://create-images-results.d-id.com/api_docs/assets/questions.png" alt="Support Image" />
|
|
134
|
-
<span style="width: 3%"><br/></span>
|
|
135
|
-
<span style="width: 60%; text-align: left;">
|
|
136
|
-
Have any questions? We are here to help! Please open an issue on GitHub and we will be happy to answer shortly.<br/><br/>
|
|
137
|
-
<a href="https://github.com/de-id/agents-sdk/issues">
|
|
138
|
-
<span style="width: 30%; text-align: center; background: #ff882eff; color: #fff; display: inline-block; padding: 6px; border-radius: 5px;">
|
|
139
|
-
Ask a question
|
|
140
|
-
</span>
|
|
141
|
-
</a>
|
|
142
|
-
</span>
|
|
143
|
-
</div>
|
|
144
|
-
|
|
145
|
-
<div style="display: flex; flex-direction: row; justify-content: space-evenly; min-height: 1px">
|
|
146
|
-
<span style="width: 100%; text-align: center; background: linear-gradient(to right, #ff882eff , #fff); color: #fff; display: inline-block; padding: 6px; border-radius: 3px;">
|
|
147
|
-
</span>
|
|
148
|
-
</div>
|
|
75
|
+
Questions, bug reports and documentation issues are welcome as [GitHub issues](https://github.com/de-id/agents-sdk/issues). The product documentation lives at [docs.d-id.com](https://docs.d-id.com).
|
|
@@ -57,7 +57,8 @@ class q extends Error {
|
|
|
57
57
|
* @example An HttpError from a request that was refused
|
|
58
58
|
* ```json
|
|
59
59
|
* {
|
|
60
|
-
* "kind": "
|
|
60
|
+
* "kind": "HttpError",
|
|
61
|
+
* "code": "InsufficientCreditsError",
|
|
61
62
|
* "message": "Account has insufficient credits",
|
|
62
63
|
* "httpStatus": 402,
|
|
63
64
|
* "endpoint": "/agt_x/chat/cht_y",
|
|
@@ -530,7 +531,7 @@ function nt(e) {
|
|
|
530
531
|
function Pe(e) {
|
|
531
532
|
return e instanceof q ? e.toJson() : { kind: "UnknownError", message: qt(e) || "UnknownError" };
|
|
532
533
|
}
|
|
533
|
-
const Zt = "3.0.0-staging.
|
|
534
|
+
const Zt = "3.0.0-staging.417", en = "https://api-js.mixpanel.com/track/?verbose=1&ip=1", St = 50, Y = [];
|
|
534
535
|
let be = !1;
|
|
535
536
|
async function Ct(e) {
|
|
536
537
|
const t = await fetch(en, {
|
|
@@ -1356,7 +1357,7 @@ async function Pn(e, t, r, a) {
|
|
|
1356
1357
|
const { version: s, ...i } = t;
|
|
1357
1358
|
switch (i.transport.provider) {
|
|
1358
1359
|
case Oe.Livekit:
|
|
1359
|
-
const { createLiveKitStreamingManager: u } = await import("./livekit-manager-
|
|
1360
|
+
const { createLiveKitStreamingManager: u } = await import("./livekit-manager-DF70NOGW.js");
|
|
1360
1361
|
return u(o, i, r);
|
|
1361
1362
|
default:
|
|
1362
1363
|
throw new Error(`Unsupported transport provider: ${i.transport.provider}`);
|
|
@@ -1600,7 +1601,7 @@ async function Wn(e, t) {
|
|
|
1600
1601
|
const P = Fe(S.avatar.type);
|
|
1601
1602
|
if (P && te(u))
|
|
1602
1603
|
throw new R(xe);
|
|
1603
|
-
P && import("./livekit-manager-
|
|
1604
|
+
P && import("./livekit-manager-DF70NOGW.js").then(({ preloadLiveKit: c }) => c()).catch(() => {
|
|
1604
1605
|
}), d.enrich(Kt(S));
|
|
1605
1606
|
const { onMessage: V, clearQueue: L } = mn(d, n, p, S, (c) => {
|
|
1606
1607
|
var m, y;
|
|
@@ -2060,3 +2061,4 @@ export {
|
|
|
2060
2061
|
Jn as v,
|
|
2061
2062
|
wt as w
|
|
2062
2063
|
};
|
|
2064
|
+
//# sourceMappingURL=index-BhhDKHW-.js.map
|