@micdrop/client 2.2.10 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,47 +1,69 @@
1
1
  # 🖐️🎤 Micdrop: Real-Time Voice Conversations with AI
2
2
 
3
- [Micdrop website](https://micdrop.dev) | [Documentation](https://micdrop.dev/docs/client) | [Demo](../../examples/demo-client)
3
+ [Micdrop website](https://micdrop.dev) | [Documentation](https://micdrop.dev/docs/client)
4
4
 
5
- Micdrop is a set of open source Typescript packages to build real-time voice conversations with AI agents. It handles all the complexities on the browser and server side (microphone, speaker, VAD, network communication, etc) and provides ready-to-use implementations for various AI providers.
5
+ Micdrop is a set of open source Typescript packages to build real-time voice conversations with AI agents. It handles all the complexities on the client and server side (microphone, speaker, VAD, network communication, etc) and provides ready-to-use implementations for various AI providers.
6
6
 
7
7
  # @micdrop/client
8
8
 
9
- The browser implementation of [Micdrop](https://micdrop.dev).
9
+ The heart of a Micdrop call, with nothing platform specific in it: the protocol, the call state, the voice activity detection, the recording and the playback scheduling.
10
10
 
11
- It is framework agnostic, you can use it with React, Vue, Angular or any other framework. See [demo-client](../../examples/demo-client) for a complete example with React.
11
+ **You install it through a platform package rather than on its own:**
12
12
 
13
- For server implementation, see [@micdrop/server](https://micdrop.dev/docs/server).
13
+ - [`@micdrop/web`](../web) for a browser
14
+ - [`@micdrop/react-native`](../react-native) for iOS and Android
14
15
 
15
- ## Features
16
+ Both re-export everything here, so `import { Micdrop } from '@micdrop/web'` gives you the same objects this package defines.
16
17
 
17
- - 🎤 Real-time microphone recording and playback
18
- - 🗣️ Voice activity detection (VAD)
19
- - 🔊 Devices selection and testing
20
- - 🔌 Full state and events for UI integration
21
- - 🌐 WebSocket-based audio streaming
18
+ ## What lives here
22
19
 
23
- ## Installation
20
+ - `MicdropClient` and the `Micdrop` singleton: the WebSocket protocol, the call state, reconnection, interruption
21
+ - `Mic` and `Speaker`: one microphone and one speaker for the app, in front of a pluggable driver
22
+ - `MicRecorder`: the reserve of audio, the chunking at 16 kHz, the wiring to the VAD
23
+ - `VolumeVAD`, `SileroVAD`, `MultipleVAD`: voice activity detection, the same on every platform
24
+ - `Pcm16AudioStream`: gapless playback of the chunks as they arrive
25
+ - `VolumeMeter`: the level the VAD and the level meters read
24
26
 
25
- ```bash
26
- npm install @micdrop/client
27
+ ## Writing a platform package
28
+
29
+ A platform provides two things, and gets everything above for free.
30
+
31
+ ```ts
32
+ import { Mic, MicDriver, Speaker, SpeakerDriver } from '@micdrop/client'
33
+
34
+ class MyMic extends MicDriver {
35
+ // start() captures, then emits Frames with mono float samples
36
+ }
37
+
38
+ class MySpeaker extends SpeakerDriver {
39
+ // play() queues 16 kHz PCM16
40
+ }
41
+
42
+ Mic.setDriver(new MyMic())
43
+ Speaker.setDriver(new MySpeaker())
27
44
  ```
28
45
 
29
- If you're using React, you can also install [@micdrop/react](https://micdrop.dev/docs/client/react-hooks) package to get a ready-to-use React hooks.
46
+ `Pcm16AudioStream` does the playback scheduling for you if the platform offers something like Web Audio: give it an `AudioSink`, which is the small slice of it that Micdrop needs.
30
47
 
31
- ## Quick Start
48
+ For Silero, provide the inference and the state machine comes from here:
32
49
 
33
- ```typescript
34
- import { Micdrop } from '@micdrop/client'
50
+ ```ts
51
+ import { setSileroModelLoader } from '@micdrop/client'
35
52
 
36
- // Start a call
37
- Micdrop.start({
38
- url: 'wss://your-server.com/call',
39
- })
53
+ setSileroModelLoader(async () => myModel) // process(frame) => probability
54
+ ```
55
+
56
+ ## Tests
57
+
58
+ The whole package runs in Node, from the audio helpers up to a full call against a real `MicdropServer`.
59
+
60
+ ```bash
61
+ pnpm --filter @micdrop/client test
40
62
  ```
41
63
 
42
64
  ## Documentation
43
65
 
44
- Read full [documentation of the Micdrop client](https://micdrop.dev/docs/client) on the [website](https://micdrop.dev).
66
+ Read the full [client documentation](https://micdrop.dev/docs/client) on the [website](https://micdrop.dev).
45
67
 
46
68
  ## License
47
69