inworld-sava 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- inworld_sava-1.0.0/.gitignore +19 -0
- inworld_sava-1.0.0/LICENSE +202 -0
- inworld_sava-1.0.0/MODEL_CARD.md +147 -0
- inworld_sava-1.0.0/NOTICE +13 -0
- inworld_sava-1.0.0/PKG-INFO +165 -0
- inworld_sava-1.0.0/README.md +127 -0
- inworld_sava-1.0.0/docs/USAGE.md +71 -0
- inworld_sava-1.0.0/docs/assets/architecture-dark.svg +136 -0
- inworld_sava-1.0.0/docs/assets/architecture-light.svg +136 -0
- inworld_sava-1.0.0/docs/assets/inworld-logo-black.svg +1 -0
- inworld_sava-1.0.0/docs/assets/inworld-logo-white.svg +1 -0
- inworld_sava-1.0.0/docs/assets/turn-dark.svg +45 -0
- inworld_sava-1.0.0/docs/assets/turn-light.svg +45 -0
- inworld_sava-1.0.0/docs/assets/vad-dark.svg +67 -0
- inworld_sava-1.0.0/docs/assets/vad-light.svg +67 -0
- inworld_sava-1.0.0/pyproject.toml +73 -0
- inworld_sava-1.0.0/src/inworld/sava/__init__.py +65 -0
- inworld_sava-1.0.0/src/inworld/sava/audio.py +65 -0
- inworld_sava-1.0.0/src/inworld/sava/checkpoint.py +86 -0
- inworld_sava-1.0.0/src/inworld/sava/cli.py +217 -0
- inworld_sava-1.0.0/src/inworld/sava/config.py +164 -0
- inworld_sava-1.0.0/src/inworld/sava/doctor.py +282 -0
- inworld_sava-1.0.0/src/inworld/sava/mel.py +154 -0
- inworld_sava-1.0.0/src/inworld/sava/model.py +251 -0
- inworld_sava-1.0.0/src/inworld/sava/modules.py +544 -0
- inworld_sava-1.0.0/src/inworld/sava/registry.py +113 -0
- inworld_sava-1.0.0/src/inworld/sava/segmenter.py +172 -0
- inworld_sava-1.0.0/src/inworld/sava/ssd.py +130 -0
- inworld_sava-1.0.0/src/inworld/sava/streaming.py +201 -0
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
__pycache__/
|
|
2
|
+
*.pyc
|
|
3
|
+
.venv/
|
|
4
|
+
dist/
|
|
5
|
+
*.egg-info/
|
|
6
|
+
.pytest_cache/
|
|
7
|
+
.ruff_cache/
|
|
8
|
+
.nfs*
|
|
9
|
+
.env
|
|
10
|
+
|
|
11
|
+
# local only: checkpoints, benchmark audio, scratch
|
|
12
|
+
models/
|
|
13
|
+
data/
|
|
14
|
+
staging/
|
|
15
|
+
*.safetensors
|
|
16
|
+
|
|
17
|
+
# bench/results holds the published scores only; other runs write elsewhere
|
|
18
|
+
bench/results/*
|
|
19
|
+
!bench/results/*.json
|
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
---
|
|
2
|
+
license: apache-2.0
|
|
3
|
+
library_name: inworld-sava
|
|
4
|
+
pipeline_tag: voice-activity-detection
|
|
5
|
+
tags: [audio, speech, voice-activity-detection, vad, semantic-vad, turn-detection, end-of-turn, streaming, realtime]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
<p align="center">
|
|
9
|
+
<picture>
|
|
10
|
+
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/inworld-logo-white.svg">
|
|
11
|
+
<img src="docs/assets/inworld-logo-black.svg" alt="Inworld" width="220">
|
|
12
|
+
</picture>
|
|
13
|
+
</p>
|
|
14
|
+
|
|
15
|
+
<h1 align="center">SAVA</h1>
|
|
16
|
+
|
|
17
|
+
<p align="center">
|
|
18
|
+
Semantic acoustic voice activity: streaming speech and end-of-turn detection in one model with an open license.
|
|
19
|
+
</p>
|
|
20
|
+
|
|
21
|
+
<p align="center">
|
|
22
|
+
<a href="https://github.com/inworld-ai/sava"><img src="https://img.shields.io/badge/GitHub-inworld--ai%2Fsava-181717?logo=github" alt="GitHub: inworld-ai/sava"></a>
|
|
23
|
+
</p>
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## TL;DR
|
|
28
|
+
|
|
29
|
+
SAVA returns two probabilities for a live audio stream:
|
|
30
|
+
* `speech` → whether a frame holds speech
|
|
31
|
+
* `turn` → whether the speaker's turn has ended
|
|
32
|
+
|
|
33
|
+
The model's shared network uses acoustic and sentence context to distinguish speech from coughs, laughs, music and keyboard clatter.
|
|
34
|
+
|
|
35
|
+
| | SAVA |
|
|
36
|
+
|---|---|
|
|
37
|
+
| made for | voice activity and conversational turn taking |
|
|
38
|
+
| input | 16 kHz mono audio |
|
|
39
|
+
| output | two probabilities every 40 ms |
|
|
40
|
+
| timing | speech probability for a frame is emitted 120 ms later (lookahead); end-of-turn probability has no lookahead |
|
|
41
|
+
| parameters | 25.5M |
|
|
42
|
+
| execution | plain PyTorch, CPU or an explicitly selected device |
|
|
43
|
+
|
|
44
|
+
Use it to open a turn when speech starts and close it when the sentence is complete.
|
|
45
|
+
The [reference turn policy](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) turns the probabilities into segments; applications can supply their own policy.
|
|
46
|
+
SAVA responds to any voice. It does not pick out a target speaker, and it leaves interruptions and backchannels to the application's dialogue policy.
|
|
47
|
+
|
|
48
|
+
## Quick start
|
|
49
|
+
|
|
50
|
+
Using a coding agent? Point it at the [setup section of AGENTS.md](https://github.com/inworld-ai/sava/blob/main/AGENTS.md#setting-up-sava-for-a-user): it installs the package, checks the machine with `sava-doctor` and runs SAVA on a first recording.
|
|
51
|
+
|
|
52
|
+
With Python 3.10 or later:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
python -m pip install inworld-sava
|
|
56
|
+
# Or from a clone: git clone https://github.com/inworld-ai/sava && cd sava && python -m pip install -e .
|
|
57
|
+
sava run recording.wav
|
|
58
|
+
sava run recording.wav --turns_dir turns/ --curves curves.json
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
```python
|
|
62
|
+
from inworld import sava
|
|
63
|
+
|
|
64
|
+
model = sava.load("inworld/sava")
|
|
65
|
+
curves = sava.run(model, sava.load_audio("recording.wav"))
|
|
66
|
+
turns = sava.segment(curves, sava.Policy(), model.lookahead_frames)
|
|
67
|
+
|
|
68
|
+
session = sava.Session(model)
|
|
69
|
+
for piece in pieces_of_16k_audio: # One-dimensional floating-point mono samples.
|
|
70
|
+
frames = session.push(piece)
|
|
71
|
+
frames = session.flush()
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
`sava.load("inworld/sava")` and the CLI's default model download the weights on first use and verify their checksum.
|
|
75
|
+
The CLI defaults to CPU; `--device` selects another device.
|
|
76
|
+
For microphone input, install `python -m pip install "inworld-sava[mic]"` and run `sava mic`.
|
|
77
|
+
|
|
78
|
+
The code, [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) and the [benchmarks](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub: [inworld-ai/sava](https://github.com/inworld-ai/sava).
|
|
79
|
+
|
|
80
|
+
## Hardware
|
|
81
|
+
|
|
82
|
+
Tested on NVIDIA Ampere, Hopper and Blackwell GPUs and on CPU.
|
|
83
|
+
On an Apple M3 Max CPU one stream takes 3.8 ms per 40 ms frame; [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) lists the measured configurations.
|
|
84
|
+
|
|
85
|
+
## The benchmarks
|
|
86
|
+
|
|
87
|
+
<p align="center">
|
|
88
|
+
<picture>
|
|
89
|
+
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/vad-dark.svg">
|
|
90
|
+
<img src="docs/assets/vad-light.svg" alt="Voice-activity frame AUC for SAVA and Silero VAD on nine public dataset readings" width="100%">
|
|
91
|
+
</picture>
|
|
92
|
+
</p>
|
|
93
|
+
|
|
94
|
+
<p align="center">
|
|
95
|
+
<picture>
|
|
96
|
+
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/turn-dark.svg">
|
|
97
|
+
<img src="docs/assets/turn-light.svg" alt="End-of-turn clip AUC for SAVA and smart-turn on public datasets" width="100%">
|
|
98
|
+
</picture>
|
|
99
|
+
</p>
|
|
100
|
+
|
|
101
|
+
Voice activity is compared with Silero VAD; end of turn is compared with pipecat smart-turn v3.2.
|
|
102
|
+
These are per-dataset measurements, with no combined ranking.
|
|
103
|
+
Speech scores use each model's own output timing, so SAVA's 120 ms lookahead makes the comparison unequal in latency.
|
|
104
|
+
Quiet speech-like noise can trigger SAVA; the ESC-50 results measure that failure mode.
|
|
105
|
+
|
|
106
|
+
[The complete tables, confidence intervals, noise tests, turn latency and reproduction steps](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub.
|
|
107
|
+
|
|
108
|
+
## Architecture
|
|
109
|
+
|
|
110
|
+
<p align="center">
|
|
111
|
+
<picture>
|
|
112
|
+
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/architecture-dark.svg">
|
|
113
|
+
<img src="docs/assets/architecture-light.svg" alt="SAVA: causal log-mel features, frame mixer, Mamba and attention trunk, speech and turn heads" width="100%">
|
|
114
|
+
</picture>
|
|
115
|
+
</p>
|
|
116
|
+
|
|
117
|
+
| part | detail |
|
|
118
|
+
|---|---|
|
|
119
|
+
| front end | causal log-mel, 128 Slaney mel bins of a 25 ms Hann DFT every 10 ms, log10 with a 1e-10 floor, `(x + 4) / 4`; four hops per frame plus the frame's log RMS |
|
|
120
|
+
| mixer | the current and previous frame's features, one linear projection to 512, RMSNorm |
|
|
121
|
+
| trunk | three Mamba-2 blocks (expand 2, head dim 64, state 128, conv kernel 4) and one attention block (8 query heads, 2 key/value heads, head dim 64, 512-frame causal window, rotary positions, RMS-normed q and k, exclusive self attention), each with a SwiGLU FFN of 2048 |
|
|
122
|
+
| voice-activity head | RMSNorm, down-projection to 512, one attention block (window 512, FFN 1536), RMSNorm, two linear readouts: on time, and the released three-frame look-ahead |
|
|
123
|
+
| end-of-turn head | the trunk output with a 64-wide embedding of the previous frame's on-time speech probability, the same block shape, a linear readout with four horizon columns (0, 200, 400, 800 ms); column 0 is the released output |
|
|
124
|
+
|
|
125
|
+
## Training data
|
|
126
|
+
|
|
127
|
+
SAVA was trained on approximately 5,000 hours of unique audio.
|
|
128
|
+
Inworld built training examples from real and synthetic conversational speech
|
|
129
|
+
mixed with non-verbal vocal sounds, noise, music and room acoustics, with a
|
|
130
|
+
speech label and an end-of-turn label for every frame. The training recipe,
|
|
131
|
+
code and data are not part of this release.
|
|
132
|
+
|
|
133
|
+
Third-party datasets used in training are credited in [NOTICE](NOTICE).
|
|
134
|
+
|
|
135
|
+
## License
|
|
136
|
+
|
|
137
|
+
Code and weights are Apache-2.0 ([LICENSE](LICENSE)).
|
|
138
|
+
Third-party notices are in [NOTICE](NOTICE).
|
|
139
|
+
|
|
140
|
+
## How to contribute
|
|
141
|
+
|
|
142
|
+
Everyone is welcome to use the model however they like and to make it better in any way they see fit!
|
|
143
|
+
Development happens on GitHub: see [CONTRIBUTING.md](https://github.com/inworld-ai/sava/blob/main/CONTRIBUTING.md) for setup, tests and bug reports.
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
<p align="center">Built for people with ❤️ by Inworld.</p>
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
Inworld SAVA
|
|
2
|
+
Copyright 2026 Inworld AI
|
|
3
|
+
|
|
4
|
+
The model weights were trained on Inworld first-party data, including data
|
|
5
|
+
purchased from vendors under private terms, and on these third-party datasets:
|
|
6
|
+
|
|
7
|
+
- pipecat smart-turn data v3.2, train split: CC BY 4.0,
|
|
8
|
+
https://huggingface.co/datasets/pipecat-ai/smart-turn-data-v3.2-train
|
|
9
|
+
- VoxConverse, dev split: CC BY 4.0, https://github.com/joonson/voxconverse
|
|
10
|
+
- VocalSound: CC BY-SA 4.0, https://github.com/YuanGongND/vocalsound
|
|
11
|
+
- Tatoeba (CC BY 2.0 FR, https://tatoeba.org), FineWeb-2 (ODC-BY 1.0,
|
|
12
|
+
https://huggingface.co/datasets/HuggingFaceFW/fineweb-2) and C4 (ODC-BY 1.0,
|
|
13
|
+
https://huggingface.co/datasets/allenai/c4), as text for synthetic speech
|
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: inworld-sava
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: SAVA, semantic acoustic voice activity: streaming voice activity and end-of-turn detection in one model.
|
|
5
|
+
Project-URL: Homepage, https://github.com/inworld-ai/sava
|
|
6
|
+
Project-URL: Benchmark, https://github.com/inworld-ai/sava/blob/main/bench/README.md
|
|
7
|
+
Author: Inworld AI
|
|
8
|
+
License-Expression: Apache-2.0
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
License-File: NOTICE
|
|
11
|
+
Keywords: end-of-turn,speech,streaming,turn-taking,voice-activity-detection
|
|
12
|
+
Classifier: Development Status :: 4 - Beta
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
|
|
16
|
+
Requires-Python: >=3.10
|
|
17
|
+
Requires-Dist: huggingface-hub
|
|
18
|
+
Requires-Dist: numpy
|
|
19
|
+
Requires-Dist: safetensors
|
|
20
|
+
Requires-Dist: soundfile
|
|
21
|
+
Requires-Dist: soxr
|
|
22
|
+
Requires-Dist: torch>=2.1
|
|
23
|
+
Provides-Extra: dev
|
|
24
|
+
Requires-Dist: isort==9.0.2; extra == 'dev'
|
|
25
|
+
Requires-Dist: pylint==4.1.1; extra == 'dev'
|
|
26
|
+
Requires-Dist: pytest; extra == 'dev'
|
|
27
|
+
Requires-Dist: scikit-learn; extra == 'dev'
|
|
28
|
+
Requires-Dist: yapf==0.43.0; extra == 'dev'
|
|
29
|
+
Provides-Extra: eval
|
|
30
|
+
Requires-Dist: onnxruntime; extra == 'eval'
|
|
31
|
+
Requires-Dist: pyarrow; extra == 'eval'
|
|
32
|
+
Requires-Dist: scikit-learn; extra == 'eval'
|
|
33
|
+
Requires-Dist: silero-vad>=6.2.0; extra == 'eval'
|
|
34
|
+
Requires-Dist: transformers; extra == 'eval'
|
|
35
|
+
Provides-Extra: mic
|
|
36
|
+
Requires-Dist: sounddevice; extra == 'mic'
|
|
37
|
+
Description-Content-Type: text/markdown
|
|
38
|
+
|
|
39
|
+
<p align="center">
|
|
40
|
+
<picture>
|
|
41
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/inworld-ai/sava/main/docs/assets/inworld-logo-white.svg">
|
|
42
|
+
<img src="https://raw.githubusercontent.com/inworld-ai/sava/main/docs/assets/inworld-logo-black.svg" alt="Inworld" width="220">
|
|
43
|
+
</picture>
|
|
44
|
+
</p>
|
|
45
|
+
|
|
46
|
+
<h1 align="center">SAVA</h1>
|
|
47
|
+
|
|
48
|
+
<p align="center">
|
|
49
|
+
Open-license (Apache-2.0) semantic acoustic voice activity: streaming speech and end-of-turn detection in one model.
|
|
50
|
+
</p>
|
|
51
|
+
|
|
52
|
+
<p align="center">
|
|
53
|
+
<a href="https://huggingface.co/inworld/sava"><img src="https://img.shields.io/badge/Hugging%20Face-inworld%2Fsava-FFD21E?logo=huggingface&logoColor=FFD21E" alt="Hugging Face: inworld/sava"></a>
|
|
54
|
+
</p>
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## TL;DR
|
|
59
|
+
|
|
60
|
+
SAVA returns two probabilities for a live audio stream:
|
|
61
|
+
* `speech` → whether a frame holds speech
|
|
62
|
+
* `turn` → whether the speaker's turn has ended
|
|
63
|
+
|
|
64
|
+
The model's shared network uses acoustic and sentence context to distinguish speech from coughs, laughs, music and keyboard clatter.
|
|
65
|
+
|
|
66
|
+
| | SAVA |
|
|
67
|
+
|---|---|
|
|
68
|
+
| made for | voice activity and conversational turn taking |
|
|
69
|
+
| input | 16 kHz mono audio |
|
|
70
|
+
| output | two probabilities every 40 ms |
|
|
71
|
+
| timing | speech probability for a frame is emitted 120 ms later (lookahead); end-of-turn probability has no lookahead |
|
|
72
|
+
| parameters | 25.5M |
|
|
73
|
+
| execution | plain PyTorch, CPU or an explicitly selected device |
|
|
74
|
+
|
|
75
|
+
Use it to open a turn when speech starts and close it when the sentence is complete.
|
|
76
|
+
The [reference turn policy](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) turns the probabilities into segments; applications can supply their own policy.
|
|
77
|
+
SAVA responds to any voice. It does not pick out a target speaker, and it leaves interruptions and backchannels to the application's dialogue policy.
|
|
78
|
+
|
|
79
|
+
## Quick start
|
|
80
|
+
|
|
81
|
+
Using a coding agent? Point it at the [setup section of AGENTS.md](https://github.com/inworld-ai/sava/blob/main/AGENTS.md#setting-up-sava-for-a-user): it installs the package, checks the machine with `sava-doctor` and runs SAVA on a first recording.
|
|
82
|
+
|
|
83
|
+
With Python 3.10 or later:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
python -m pip install inworld-sava
|
|
87
|
+
# Or from a clone: git clone https://github.com/inworld-ai/sava && cd sava && python -m pip install -e .
|
|
88
|
+
sava run recording.wav
|
|
89
|
+
sava run recording.wav --turns_dir turns/ --curves curves.json
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
```python
|
|
93
|
+
from inworld import sava
|
|
94
|
+
|
|
95
|
+
model = sava.load("inworld/sava")
|
|
96
|
+
curves = sava.run(model, sava.load_audio("recording.wav"))
|
|
97
|
+
turns = sava.segment(curves, sava.Policy(), model.lookahead_frames)
|
|
98
|
+
|
|
99
|
+
session = sava.Session(model)
|
|
100
|
+
for piece in pieces_of_16k_audio: # One-dimensional floating-point mono samples.
|
|
101
|
+
frames = session.push(piece)
|
|
102
|
+
frames = session.flush()
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
The weights download from [huggingface.co/inworld/sava](https://huggingface.co/inworld/sava) on first use: `sava.load("inworld/sava")` and the CLI's default model fetch the registered file and check its checksum, and both also take a local checkpoint path or an `hf://` location.
|
|
106
|
+
The current checkpoint's checksum is in [the registry](https://github.com/inworld-ai/sava/blob/main/src/inworld/sava/registry.py).
|
|
107
|
+
|
|
108
|
+
The CLI defaults to CPU; `--device` selects another device.
|
|
109
|
+
For microphone input, install `python -m pip install "inworld-sava[mic]"` and run `sava mic`.
|
|
110
|
+
[The integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) covers streaming alignment, flushing, the turn policy, limits and tested configurations.
|
|
111
|
+
|
|
112
|
+
## Hardware
|
|
113
|
+
|
|
114
|
+
Tested on NVIDIA Ampere, Hopper and Blackwell GPUs and on CPU.
|
|
115
|
+
On an Apple M3 Max CPU one stream takes 3.8 ms per 40 ms frame; [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) lists the measured configurations.
|
|
116
|
+
|
|
117
|
+
## The benchmarks
|
|
118
|
+
|
|
119
|
+
<p align="center">
|
|
120
|
+
<picture>
|
|
121
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/inworld-ai/sava/main/docs/assets/vad-dark.svg">
|
|
122
|
+
<img src="https://raw.githubusercontent.com/inworld-ai/sava/main/docs/assets/vad-light.svg" alt="Voice-activity frame AUC for SAVA and Silero VAD on nine public dataset readings" width="100%">
|
|
123
|
+
</picture>
|
|
124
|
+
</p>
|
|
125
|
+
|
|
126
|
+
<p align="center">
|
|
127
|
+
<picture>
|
|
128
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/inworld-ai/sava/main/docs/assets/turn-dark.svg">
|
|
129
|
+
<img src="https://raw.githubusercontent.com/inworld-ai/sava/main/docs/assets/turn-light.svg" alt="End-of-turn clip AUC for SAVA and smart-turn on public datasets" width="100%">
|
|
130
|
+
</picture>
|
|
131
|
+
</p>
|
|
132
|
+
|
|
133
|
+
Voice activity is compared with Silero VAD; end of turn is compared with pipecat smart-turn v3.2.
|
|
134
|
+
These are per-dataset measurements, with no combined ranking.
|
|
135
|
+
Speech scores use each model's own output timing, so SAVA's 120 ms lookahead makes the comparison unequal in latency.
|
|
136
|
+
Quiet speech-like noise can trigger SAVA; the ESC-50 results measure that failure mode.
|
|
137
|
+
|
|
138
|
+
Complete tables, confidence intervals, noise tests, turn latency and reproduction steps are in [bench/README.md](https://github.com/inworld-ai/sava/blob/main/bench/README.md), with the [result JSONs](https://github.com/inworld-ai/sava/tree/main/bench/results) and [dataset manifests](https://github.com/inworld-ai/sava/blob/main/bench/DATA.md).
|
|
139
|
+
Figures are generated from those JSONs by `python docs/build_figures.py`.
|
|
140
|
+
|
|
141
|
+
## Architecture
|
|
142
|
+
|
|
143
|
+
<p align="center">
|
|
144
|
+
<picture>
|
|
145
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/inworld-ai/sava/main/docs/assets/architecture-dark.svg">
|
|
146
|
+
<img src="https://raw.githubusercontent.com/inworld-ai/sava/main/docs/assets/architecture-light.svg" alt="SAVA: causal log-mel features, frame mixer, Mamba and attention trunk, speech and turn heads" width="100%">
|
|
147
|
+
</picture>
|
|
148
|
+
</p>
|
|
149
|
+
|
|
150
|
+
See the model card for [architecture details](https://github.com/inworld-ai/sava/blob/main/MODEL_CARD.md#architecture) and [training data](https://github.com/inworld-ai/sava/blob/main/MODEL_CARD.md#training-data).
|
|
151
|
+
|
|
152
|
+
## License
|
|
153
|
+
|
|
154
|
+
Code and weights are Apache-2.0 ([LICENSE](https://github.com/inworld-ai/sava/blob/main/LICENSE)).
|
|
155
|
+
Third-party notices are in [NOTICE](https://github.com/inworld-ai/sava/blob/main/NOTICE).
|
|
156
|
+
|
|
157
|
+
## How to contribute
|
|
158
|
+
|
|
159
|
+
Contributions are welcome: ask your agent to open a PR and we'll take a look.
|
|
160
|
+
See [CONTRIBUTING.md](https://github.com/inworld-ai/sava/blob/main/CONTRIBUTING.md) for setup, tests and bug reports, and [AGENTS.md](https://github.com/inworld-ai/sava/blob/main/AGENTS.md) for coding conventions.
|
|
161
|
+
The package lives in `src/inworld/sava/`, the benchmark harness in `bench/`, and mirrored unit and golden-parity tests in `tests/`.
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
<p align="center">Built for people with ❤️ by Inworld.</p>
|