behavioralsignals 0.7.0__tar.gz → 0.7.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {behavioralsignals-0.7.0/src/behavioralsignals.egg-info → behavioralsignals-0.7.2}/PKG-INFO +288 -192
- behavioralsignals-0.7.2/README.md +482 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/pyproject.toml +1 -1
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/mcp_server.py +14 -4
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/models.py +10 -6
- behavioralsignals-0.7.2/src/behavioralsignals/utils.py +45 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2/src/behavioralsignals.egg-info}/PKG-INFO +288 -192
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/tests/test_mcp_server.py +17 -5
- behavioralsignals-0.7.2/tests/test_utils.py +77 -0
- behavioralsignals-0.7.0/README.md +0 -386
- behavioralsignals-0.7.0/src/behavioralsignals/utils.py +0 -24
- behavioralsignals-0.7.0/tests/test_utils.py +0 -25
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/LICENSE +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/setup.cfg +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/__init__.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/base.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/behavioral.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/client.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/configuration.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/deepfakes.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/generated/__init__.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/generated/api_pb2.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/generated/api_pb2.pyi +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/generated/api_pb2_grpc.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals/py.typed +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals.egg-info/SOURCES.txt +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals.egg-info/dependency_links.txt +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals.egg-info/entry_points.txt +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals.egg-info/requires.txt +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/src/behavioralsignals.egg-info/top_level.txt +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/tests/test_client.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/tests/test_models.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/tests/test_package.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/tests/test_streaming.py +0 -0
- {behavioralsignals-0.7.0 → behavioralsignals-0.7.2}/tests/test_wait_for_result.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: behavioralsignals
|
|
3
|
-
Version: 0.7.
|
|
3
|
+
Version: 0.7.2
|
|
4
4
|
Summary: Official Python SDK for the Behavioral Signals API: voice emotion/behavior analysis and audio/video deepfake detection
|
|
5
5
|
Author: Behavioral Signal Technologies
|
|
6
6
|
License: Apache License
|
|
@@ -252,8 +252,6 @@ Dynamic: license-file
|
|
|
252
252
|
|
|
253
253
|
<div align="center">
|
|
254
254
|
|
|
255
|
-
|
|
256
|
-
|
|
257
255
|
[](https://discord.com/invite/fxjRrbMH3Q)
|
|
258
256
|
[](https://x.com/behaviorsignals)
|
|
259
257
|
[](https://behavioralsignals.readme.io/)
|
|
@@ -271,106 +269,85 @@ Dynamic: license-file
|
|
|
271
269
|
|
|
272
270
|
</div>
|
|
273
271
|
|
|
274
|
-
Python SDK for the Behavioral Signals API.
|
|
275
|
-
Our API enables developers to integrate behavioral analysis into their applications, both in batch and streaming modes.
|
|
276
|
-
See the [API documentation](https://behavioralsignals.readme.io/) for details. It is also available as [llms.txt](https://behavioralsignals.readme.io/llms.txt) for AI coding assistants.
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
## Table of Contents
|
|
280
|
-
* [Behavioral Signals API Python SDK](#behavioral-signals-api-python-sdk)
|
|
281
|
-
* [Features](#features)
|
|
282
|
-
* [Requirements](#requirements)
|
|
283
|
-
* [API Key Setup](#api-key-setup)
|
|
284
|
-
* [SDK Installation](#sdk-installation)
|
|
285
|
-
* [SDK Example Usage](#sdk-example-usage)
|
|
286
|
-
* [Behavioral API Batch Mode](#behavioral-api-batch-mode)
|
|
287
|
-
* [Behavioral API Streaming Mode](#behavioral-api-streaming-mode)
|
|
288
|
-
* [Deepfakes API Batch Mode](#deepfakes-api-batch-mode)
|
|
289
|
-
* [Deepfakes API Streaming Mode](#deepfakes-api-streaming-mode)
|
|
290
|
-
* [Available Methods](#available-methods)
|
|
291
|
-
* [Error Handling](#error-handling)
|
|
292
|
-
* [Use with AI assistants (MCP)](#use-with-ai-assistants-mcp)
|
|
293
|
-
* [Running the MCP server manually](#running-the-mcp-server-manually)
|
|
294
|
-
|
|
295
|
-
## Features
|
|
296
|
-
|
|
297
|
-
- **Behavioral Analysis API** : Analyze human behavior in both batch (offline) and streaming (online) modes.
|
|
272
|
+
Official Python SDK for the [Behavioral Signals API](https://behavioralsignals.readme.io/).
|
|
298
273
|
|
|
299
|
-
|
|
300
|
-
- Supports batch (offline) and streaming (online) modes
|
|
301
|
-
- Compatible with a wide range of spoken languages
|
|
274
|
+
Estimate behavioral signals from speech and detect deepfake speech using batch and real-time audio APIs. Experimental video deepfake detection is also available in batch mode.
|
|
302
275
|
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
276
|
+
[Python SDK Documentation](https://behavioralsignals.readme.io/docs/behavioral-signals-python-sdk) ·
|
|
277
|
+
[Examples](examples/) ·
|
|
278
|
+
[PyPI](https://pypi.org/project/behavioralsignals/) ·
|
|
279
|
+
[Contributing](CONTRIBUTING.md)
|
|
307
280
|
|
|
308
|
-
|
|
281
|
+
## Features
|
|
309
282
|
|
|
310
|
-
|
|
283
|
+
- **Behavioral Analysis** — estimate behavioral signals from speech in batch and real-time streaming modes
|
|
284
|
+
- **Deepfake Detection** — detect synthetic or manipulated speech in batch and real-time streaming modes
|
|
285
|
+
- **Video Deepfake Detection (Experimental, Batch Only)** — analyze both the video frames and audio track of supported video files
|
|
286
|
+
- **Core Speech Attributes (Batch Only)** — automatic speech recognition (ASR), speaker diarization, and language identification
|
|
287
|
+
- **Embeddings** — retrieve speaker and behavioral embeddings from the Behavioral API, or speaker and deepfake embeddings from the Deepfakes API
|
|
288
|
+
- **S3 Input** — submit audio and video using S3 presigned URLs
|
|
289
|
+
- **MCP Server** — use Behavioral Signals from MCP-compatible AI assistants
|
|
311
290
|
|
|
312
|
-
|
|
313
|
-
* `ffmpeg`,
|
|
314
|
-
* Python dependencies as specified in `pyproject.toml`
|
|
291
|
+
## Quickstart
|
|
315
292
|
|
|
293
|
+
### Install
|
|
316
294
|
|
|
317
|
-
|
|
295
|
+
```bash
|
|
296
|
+
pip install behavioralsignals
|
|
297
|
+
```
|
|
318
298
|
|
|
319
|
-
|
|
299
|
+
Requires Python 3.10 or later.
|
|
320
300
|
|
|
321
|
-
|
|
301
|
+
### Configure credentials
|
|
322
302
|
|
|
323
|
-
|
|
324
|
-
export BEHAVIORALSIGNALS_CID=your_cid
|
|
325
|
-
export BEHAVIORALSIGNALS_API_KEY=your_api_key
|
|
326
|
-
```
|
|
303
|
+
Create an account and API key in the [Behavioral Signals portal](https://portal.behavioralsignals.com/).
|
|
327
304
|
|
|
328
|
-
|
|
305
|
+
Set your client ID (CID) and API key as environment variables:
|
|
329
306
|
|
|
330
307
|
```bash
|
|
331
|
-
|
|
308
|
+
export BEHAVIORALSIGNALS_CID="your_cid"
|
|
309
|
+
export BEHAVIORALSIGNALS_API_KEY="your_api_key"
|
|
332
310
|
```
|
|
333
311
|
|
|
334
|
-
|
|
312
|
+
The SDK will pick them up automatically:
|
|
335
313
|
|
|
336
|
-
|
|
337
|
-
|
|
314
|
+
```python
|
|
315
|
+
from behavioralsignals import Client
|
|
338
316
|
|
|
339
|
-
|
|
340
|
-
|
|
317
|
+
client = Client()
|
|
318
|
+
```
|
|
341
319
|
|
|
342
|
-
|
|
343
|
-
You can also find more detailed examples for both [batch](https://github.com/BehavioralSignalTechnologies/behavioralsignals-python/blob/main/examples/batch/README.md) and [streaming](https://github.com/BehavioralSignalTechnologies/behavioralsignals-python/blob/main/examples/streaming/README.md) in the `examples/` directory.
|
|
320
|
+
You can also pass the credentials directly:
|
|
344
321
|
|
|
345
|
-
|
|
322
|
+
```python
|
|
323
|
+
client = Client("your_cid", "your_api_key")
|
|
324
|
+
```
|
|
346
325
|
|
|
347
|
-
|
|
348
|
-
`wait_for_result` checks the process status until processing is complete, then returns the results (available since version 0.6.0).
|
|
349
|
-
Pass `timeout` in seconds to stop waiting after that long; it raises `TimeoutError` if processing has not finished.
|
|
326
|
+
### Analyze audio
|
|
350
327
|
|
|
351
328
|
```python
|
|
352
329
|
from behavioralsignals import Client
|
|
330
|
+
from behavioralsignals.utils import print_results
|
|
353
331
|
|
|
354
|
-
client = Client(
|
|
332
|
+
client = Client()
|
|
355
333
|
|
|
356
|
-
|
|
357
|
-
|
|
334
|
+
process = client.behavioral.upload_audio(file_path="audio.wav")
|
|
335
|
+
result = client.behavioral.wait_for_result(
|
|
336
|
+
pid=process.pid,
|
|
337
|
+
timeout=600,
|
|
338
|
+
)
|
|
358
339
|
|
|
359
|
-
|
|
360
|
-
top = item.prediction[0]
|
|
361
|
-
label = item.finalLabel or top.score # continuous tasks (e.g. intensity) have no label
|
|
362
|
-
if not label:
|
|
363
|
-
continue # the features row carries embeddings, not a result
|
|
364
|
-
confidence = f" ({float(top.posterior):.1%})" if top.posterior else ""
|
|
365
|
-
print(f"{item.st} {item.et} {item.task} {label}{confidence}")
|
|
340
|
+
print_results(result.results)
|
|
366
341
|
```
|
|
367
342
|
|
|
368
|
-
|
|
343
|
+
`upload_audio()` returns a process with a unique process ID (`pid`). `wait_for_result()` polls until processing completes and returns the analysis result.
|
|
344
|
+
|
|
345
|
+
`print_results()` prints one line per result: the start and end time in seconds, the task, and the final label with its probability.
|
|
369
346
|
Continuous tasks such as `intensity` have no label, only a score, and `asr` and `diarization` have a label without a probability.
|
|
370
347
|
For a 10-second clip of one speaker, the output starts like this:
|
|
371
348
|
|
|
372
349
|
```
|
|
373
|
-
0.487 3.001 asr
|
|
350
|
+
0.487 3.001 asr The birch canoe slid on the smooth plank.
|
|
374
351
|
0.487 3.001 diarization SPEAKER_00
|
|
375
352
|
0.487 3.001 language en (98.9%)
|
|
376
353
|
0.487 3.001 gender female (99.7%)
|
|
@@ -380,191 +357,269 @@ For a 10-second clip of one speaker, the output starts like this:
|
|
|
380
357
|
0.487 3.001 intensity 0.0873
|
|
381
358
|
```
|
|
382
359
|
|
|
383
|
-
Each row also has `prediction`, the list of all labels with their probabilities. Use `
|
|
360
|
+
Each row also has `prediction`, the list of all labels with their probabilities. Use `result.model_dump()` to get the results as a dictionary.
|
|
384
361
|
|
|
385
|
-
|
|
362
|
+
To include speaker and behavioral embeddings:
|
|
386
363
|
|
|
387
364
|
```python
|
|
388
|
-
|
|
389
|
-
|
|
365
|
+
process = client.behavioral.upload_audio(
|
|
366
|
+
file_path="audio.wav",
|
|
367
|
+
embeddings=True,
|
|
368
|
+
)
|
|
390
369
|
```
|
|
391
370
|
|
|
392
|
-
|
|
393
|
-
You can change these when creating the client, or pass `None` for no limit:
|
|
371
|
+
### Behavioral outputs
|
|
394
372
|
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
373
|
+
The Behavioral API returns **24 signals** across 8 dimensions: 18 behavioral signals and 6 speaker signals. Each label is its own signal with its own probability.
|
|
374
|
+
All eight dimensions are scored together on every utterance longer than 1 second, so one utterance can be *sad + weak + negative + slow + hesitating + withdrawn*, with a voice estimated as female and aged 31 - 45. That gives 4 × 3 × 3 × 3 × 2 × 3 × 2 × 4 = **5,184 possible profiles** per utterance.
|
|
375
|
+
|
|
376
|
+
| Task | What it measures | Labels | # |
|
|
377
|
+
|---|---|---|---|
|
|
378
|
+
| `emotion` | Basic emotion in the voice | `happy`, `angry`, `sad`, `neutral` | 4 |
|
|
379
|
+
| `strength` | Arousal: energy in the voice | `strong`, `weak`, `neutral` | 3 |
|
|
380
|
+
| `positivity` | Valence: sentiment of the tone | `positive`, `negative`, `neutral` | 3 |
|
|
381
|
+
| `speaking_rate` | How fast the speaker talks, compared to speakers in general, not to their own pace | `fast`, `slow`, `normal` | 3 |
|
|
382
|
+
| `hesitation` | Signs of hesitation | `yes`, `no` | 2 |
|
|
383
|
+
| `engagement` | Involved or detached tone | `engaged`, `withdrawn`, `neutral` | 3 |
|
|
384
|
+
| `gender` | Sex of the speaker, estimated from the voice | `female`, `male` | 2 |
|
|
385
|
+
| `age` | Age range of the speaker, estimated from the voice | `18 - 22`, `23 - 30`, `31 - 45`, `46 - 65` | 4 |
|
|
386
|
+
| **Total** | | | **24** |
|
|
387
|
+
|
|
388
|
+
Plus `intensity`: how intense the emotion is, as one score between 0 and 1. It is not a score for any one emotion, even when `emotion` is `angry`. It has no labels and is not counted in the 24.
|
|
389
|
+
In batch results, utterances of 1 second or less return only `diarization`, `asr` and `language`. When streaming, every segment gets all 24 signals and `intensity`, however short it is.
|
|
390
|
+
In batch results, `gender` and `age` are given per speaker: every utterance of a speaker longer than 1 second gets the same values, averaged over the file. When streaming, `gender` and `age` are estimated on each segment.
|
|
391
|
+
The other tasks have open-ended values, not fixed labels: `diarization` (a speaker ID such as `SPEAKER_00`), `asr` (the transcript) and `language`. With `embeddings=True`, `features` holds the behavioral embedding.
|
|
392
|
+
All results of one utterance share its `id`, `startTime` and `endTime`. To split results by speaker, use the `diarization` label of each utterance. Speaker IDs are generic: the API does not say who is the agent and who is the customer.
|
|
398
393
|
|
|
399
|
-
|
|
400
|
-
If an upload times out, the server may still have received the file and started a process, so check `list_processes()` (or `list_video_processes()` for videos) before uploading again.
|
|
394
|
+
`neutral` means something different in each task, so always read a label together with its task: `emotion: neutral` means no clear emotion, `strength: neutral` normal energy, `positivity: neutral` neither positive nor negative, and `engagement: neutral` neither engaged nor withdrawn.
|
|
401
395
|
|
|
402
|
-
|
|
396
|
+
`finalLabel` gives one answer per task. For behavioral tasks it is the label with the highest probability.
|
|
397
|
+
When you need a score instead of a label, use the probabilities in `prediction`. They are strings, so convert them with `float()`.
|
|
398
|
+
For each task in the table they add up to 1, so each label's probability is a score from 0 to 1 that you can track over time, average over a call, or compare against your own threshold.
|
|
399
|
+
For example, the `angry` probability of `emotion` is a score for anger. `intensity` is not: it does not say which emotion is intense.
|
|
403
400
|
|
|
404
|
-
|
|
401
|
+
`positivity` is valence and `strength` is arousal, the two dimensions often used in emotion research. The API gives them as probabilities from 0 to 1. To put them on the usual scale from -1 to 1, subtract the probabilities of the two opposite labels:
|
|
405
402
|
|
|
406
403
|
```python
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
for result in client.behavioral.stream_audio(audio_stream=audio_stream, options=options):
|
|
415
|
-
for item in result.results or []:
|
|
416
|
-
top = item.prediction[0]
|
|
417
|
-
label = item.finalLabel or top.score # continuous tasks (e.g. intensity) have no label
|
|
418
|
-
if not label:
|
|
419
|
-
continue # the features row carries embeddings, not a result
|
|
420
|
-
confidence = f" ({float(top.posterior):.1%})" if top.posterior else ""
|
|
421
|
-
print(f"{item.st} {item.et} {item.task} {label}{confidence}")
|
|
404
|
+
for item in result.results:
|
|
405
|
+
if item.task in ("positivity", "strength"):
|
|
406
|
+
scores = {p.label: float(p.posterior) for p in item.prediction}
|
|
407
|
+
if item.task == "positivity":
|
|
408
|
+
print(item.st, "valence", scores["positive"] - scores["negative"])
|
|
409
|
+
else:
|
|
410
|
+
print(item.st, "arousal", scores["strong"] - scores["weak"])
|
|
422
411
|
```
|
|
423
412
|
|
|
424
|
-
|
|
413
|
+
All probabilities are model estimates: use them to compare utterances and follow trends, not as exact measurements.
|
|
414
|
+
|
|
415
|
+
See [Definition of behaviors](https://behavioralsignals.readme.io/docs/definition-of-behaviors) for what each signal means.
|
|
425
416
|
|
|
426
|
-
|
|
417
|
+
## Deepfake Detection
|
|
418
|
+
|
|
419
|
+
### Audio
|
|
420
|
+
|
|
421
|
+
Batch deepfake detection follows the same workflow as Behavioral Analysis:
|
|
427
422
|
|
|
428
423
|
```python
|
|
429
424
|
from behavioralsignals import Client
|
|
425
|
+
from behavioralsignals.utils import print_results
|
|
430
426
|
|
|
431
|
-
client = Client(
|
|
427
|
+
client = Client()
|
|
432
428
|
|
|
433
|
-
|
|
434
|
-
|
|
429
|
+
process = client.deepfakes.upload_audio(file_path="audio.wav")
|
|
430
|
+
result = client.deepfakes.wait_for_result(
|
|
431
|
+
pid=process.pid,
|
|
432
|
+
timeout=600,
|
|
433
|
+
)
|
|
435
434
|
|
|
436
|
-
|
|
437
|
-
top = item.prediction[0]
|
|
438
|
-
label = item.finalLabel or top.score # some tasks have no label, only a score
|
|
439
|
-
if not label:
|
|
440
|
-
continue # the features row carries embeddings, not a result
|
|
441
|
-
confidence = f" ({float(top.posterior):.1%})" if top.posterior else ""
|
|
442
|
-
print(f"{item.st} {item.et} {item.task} {label}{confidence}")
|
|
435
|
+
print_results(result.results)
|
|
443
436
|
```
|
|
444
437
|
|
|
445
|
-
|
|
438
|
+
To include speaker and deepfake embeddings:
|
|
446
439
|
|
|
447
440
|
```python
|
|
448
|
-
|
|
449
|
-
|
|
441
|
+
process = client.deepfakes.upload_audio(
|
|
442
|
+
file_path="audio.wav",
|
|
443
|
+
embeddings=True,
|
|
444
|
+
)
|
|
450
445
|
```
|
|
451
446
|
|
|
447
|
+
### Experimental Generator Detection
|
|
452
448
|
|
|
453
|
-
|
|
449
|
+
Generator detection is an experimental batch feature that attempts to identify the model used to generate deepfake audio.
|
|
454
450
|
|
|
455
|
-
|
|
456
|
-
|
|
451
|
+
```python
|
|
452
|
+
process = client.deepfakes.upload_audio(
|
|
453
|
+
file_path="audio.wav",
|
|
454
|
+
enable_generator_detection=True,
|
|
455
|
+
)
|
|
456
|
+
```
|
|
457
457
|
|
|
458
|
-
|
|
458
|
+
See the [generator detection documentation](https://behavioralsignals.readme.io/docs/generator-detection) for details and supported generators.
|
|
459
459
|
|
|
460
|
-
|
|
461
|
-
from behavioralsignals import Client
|
|
460
|
+
### Experimental Video Deepfake Detection
|
|
462
461
|
|
|
463
|
-
|
|
462
|
+
Video deepfake detection is available in **batch mode only** and is currently experimental.
|
|
464
463
|
|
|
465
|
-
|
|
466
|
-
|
|
464
|
+
```python
|
|
465
|
+
process = client.deepfakes.upload_video(file_path="video.mp4")
|
|
466
|
+
result = client.deepfakes.wait_for_video_result(
|
|
467
|
+
pid=process.pid,
|
|
468
|
+
timeout=600,
|
|
469
|
+
)
|
|
467
470
|
```
|
|
468
471
|
|
|
469
|
-
|
|
472
|
+
Unlike an audio result, a video result contains two separate lists:
|
|
470
473
|
|
|
471
|
-
|
|
474
|
+
- `audio_results` — deepfake detection results for the video's audio track
|
|
475
|
+
- `video_results` — deepfake detection results for the video frames
|
|
472
476
|
|
|
473
|
-
|
|
477
|
+
For example:
|
|
474
478
|
|
|
475
479
|
```python
|
|
476
|
-
|
|
480
|
+
print_results(result.video_results)
|
|
481
|
+
```
|
|
477
482
|
|
|
478
|
-
|
|
483
|
+
You can also submit a video using an S3 presigned URL with `upload_s3_presigned_video_url()` and inspect video processes using `list_video_processes()` and `get_video_process()`.
|
|
479
484
|
|
|
480
|
-
|
|
481
|
-
output = client.deepfakes.wait_for_video_result(pid=response.pid, timeout=600)
|
|
482
|
-
```
|
|
485
|
+
See the [video deepfake documentation](https://behavioralsignals.readme.io/docs/submit-a-file-for-processing) for supported formats, limits, and the current experimental status.
|
|
483
486
|
|
|
484
|
-
|
|
487
|
+
## Streaming
|
|
488
|
+
|
|
489
|
+
Both the Behavioral and Deepfakes APIs support real-time **audio** streaming over gRPC.
|
|
490
|
+
|
|
491
|
+
To stream an audio file:
|
|
485
492
|
|
|
486
493
|
```python
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
+
from behavioralsignals import Client, StreamingOptions
|
|
495
|
+
from behavioralsignals.utils import make_audio_stream, print_results
|
|
496
|
+
|
|
497
|
+
client = Client()
|
|
498
|
+
|
|
499
|
+
audio_stream, sample_rate = make_audio_stream(
|
|
500
|
+
"audio.wav",
|
|
501
|
+
chunk_size=0.25,
|
|
502
|
+
)
|
|
503
|
+
|
|
504
|
+
options = StreamingOptions(
|
|
505
|
+
sample_rate=sample_rate,
|
|
506
|
+
encoding="LINEAR_PCM",
|
|
507
|
+
)
|
|
508
|
+
|
|
509
|
+
for result in client.behavioral.stream_audio(
|
|
510
|
+
audio_stream=audio_stream,
|
|
511
|
+
options=options,
|
|
512
|
+
):
|
|
513
|
+
print_results(result.results)
|
|
494
514
|
```
|
|
495
515
|
|
|
496
|
-
|
|
516
|
+
`chunk_size` is specified in **seconds**, so `0.25` corresponds to 250 ms.
|
|
497
517
|
|
|
498
|
-
|
|
518
|
+
`stream_audio()` can also accept your own `Iterator[bytes]`, for example from a microphone or live call.
|
|
499
519
|
|
|
500
|
-
|
|
520
|
+
For real-time deepfake detection, use the same interface:
|
|
501
521
|
|
|
502
522
|
```python
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
options = StreamingOptions(sample_rate=sample_rate, encoding="LINEAR_PCM")
|
|
509
|
-
|
|
510
|
-
for result in client.deepfakes.stream_audio(audio_stream=audio_stream, options=options):
|
|
511
|
-
for item in result.results or []:
|
|
512
|
-
top = item.prediction[0]
|
|
513
|
-
label = item.finalLabel or top.score # some tasks have no label, only a score
|
|
514
|
-
if not label:
|
|
515
|
-
continue # the features row carries embeddings, not a result
|
|
516
|
-
confidence = f" ({float(top.posterior):.1%})" if top.posterior else ""
|
|
517
|
-
print(f"{item.st} {item.et} {item.task} {label}{confidence}")
|
|
523
|
+
for result in client.deepfakes.stream_audio(
|
|
524
|
+
audio_stream=audio_stream,
|
|
525
|
+
options=options,
|
|
526
|
+
):
|
|
527
|
+
print_results(result.results)
|
|
518
528
|
```
|
|
519
529
|
|
|
530
|
+
See the [streaming examples](examples/streaming/) and [streaming documentation](https://behavioralsignals.readme.io/docs/streaming-using-python-sdk) for more.
|
|
531
|
+
|
|
520
532
|
## Available Methods
|
|
521
533
|
|
|
522
|
-
`client.behavioral` and `client.deepfakes`
|
|
534
|
+
`client.behavioral` and `client.deepfakes` expose the same methods for audio. Video methods are available only on `client.deepfakes`.
|
|
523
535
|
|
|
524
536
|
| Method | What it does |
|
|
525
537
|
|---|---|
|
|
526
|
-
| `upload_audio(file_path, ...)` | Uploads an audio file and returns the process,
|
|
527
|
-
| `upload_s3_presigned_url(url, ...)` |
|
|
538
|
+
| `upload_audio(file_path, ...)` | Uploads an audio file and returns the process, including its `pid` |
|
|
539
|
+
| `upload_s3_presigned_url(url, ...)` | Submits audio using an S3 presigned URL |
|
|
528
540
|
| `wait_for_result(pid, timeout=None)` | Waits for a process to finish and returns its results |
|
|
529
541
|
| `get_result(pid)` | Returns the results of a finished process |
|
|
530
|
-
| `get_process(pid)` | Returns a process and its status |
|
|
531
|
-
| `list_processes(
|
|
532
|
-
| `stream_audio(audio_stream, options)` |
|
|
542
|
+
| `get_process(pid)` | Returns a process and its current status |
|
|
543
|
+
| `list_processes(...)` | Lists audio processes |
|
|
544
|
+
| `stream_audio(audio_stream, options)` | Streams audio and yields results as they arrive |
|
|
533
545
|
| `upload_video(file_path, ...)` | Deepfakes only: uploads a video file |
|
|
534
|
-
| `upload_s3_presigned_video_url(url, ...)` | Deepfakes only:
|
|
546
|
+
| `upload_s3_presigned_video_url(url, ...)` | Deepfakes only: submits video using an S3 presigned URL |
|
|
535
547
|
| `wait_for_video_result(pid, timeout=None)` | Deepfakes only: waits for a video process to finish and returns its results |
|
|
536
548
|
| `get_video_result(pid)` | Deepfakes only: returns the results of a finished video process |
|
|
537
549
|
| `get_video_process(pid)` | Deepfakes only: returns a video process and its status |
|
|
538
|
-
| `list_video_processes(...)` | Deepfakes only: lists
|
|
550
|
+
| `list_video_processes(...)` | Deepfakes only: lists video processes |
|
|
551
|
+
|
|
552
|
+
`behavioralsignals.utils` also has two helpers: `make_audio_stream(file_path, chunk_size=0.25)` turns an audio file into chunks for `stream_audio()`, and `print_results(items)` prints result rows as shown in [Analyze audio](#analyze-audio).
|
|
539
553
|
|
|
540
|
-
|
|
554
|
+
For detailed parameters and result schemas, see the [full API documentation](https://behavioralsignals.readme.io/).
|
|
541
555
|
|
|
542
|
-
|
|
543
|
-
|
|
556
|
+
## Timeouts and Error Handling
|
|
557
|
+
|
|
558
|
+
### Batch API
|
|
559
|
+
|
|
560
|
+
Batch HTTP API errors raise `BehavioralSignalsError`. Its `status_code` contains the HTTP status code.
|
|
544
561
|
|
|
545
562
|
```python
|
|
546
563
|
from behavioralsignals import BehavioralSignalsError, Client
|
|
547
564
|
|
|
548
|
-
client = Client(
|
|
565
|
+
client = Client()
|
|
549
566
|
|
|
550
567
|
try:
|
|
551
|
-
|
|
568
|
+
result = client.behavioral.get_result(pid=12345)
|
|
552
569
|
except BehavioralSignalsError as error:
|
|
553
570
|
print(error.status_code, error)
|
|
554
571
|
```
|
|
555
572
|
|
|
556
|
-
`
|
|
573
|
+
`wait_for_result()` and `wait_for_video_result()` also raise:
|
|
574
|
+
|
|
575
|
+
- `TimeoutError` if processing has not finished within the supplied `timeout`
|
|
576
|
+
- `RuntimeError` if the process finishes unsuccessfully, for example because of insufficient credits
|
|
577
|
+
|
|
557
578
|
Network problems raise the usual `requests` exceptions, such as `requests.ConnectionError`.
|
|
558
579
|
|
|
559
|
-
|
|
580
|
+
Each HTTP request also has its own timeout. By default, the SDK allows 10 seconds to connect and up to 60 seconds waiting for the server, or 300 seconds for uploads.
|
|
581
|
+
|
|
582
|
+
You can customize these limits when creating the client:
|
|
583
|
+
|
|
584
|
+
```python
|
|
585
|
+
client = Client(
|
|
586
|
+
timeout=(10, 120),
|
|
587
|
+
upload_timeout=(10, 900),
|
|
588
|
+
)
|
|
589
|
+
```
|
|
590
|
+
|
|
591
|
+
These request timeouts are separate from the overall `timeout` passed to `wait_for_result()`.
|
|
592
|
+
|
|
593
|
+
> If an upload request times out, the server may still have received the file and started processing it. Check `list_processes()` — or `list_video_processes()` for videos — before uploading the same file again.
|
|
594
|
+
|
|
595
|
+
### Streaming API
|
|
596
|
+
|
|
597
|
+
Streaming uses gRPC rather than the batch HTTP API. Streaming failures can therefore raise `grpc.RpcError` instead of `BehavioralSignalsError`.
|
|
598
|
+
|
|
599
|
+
```python
|
|
600
|
+
import grpc
|
|
601
|
+
|
|
602
|
+
try:
|
|
603
|
+
for result in client.behavioral.stream_audio(
|
|
604
|
+
audio_stream=audio_stream,
|
|
605
|
+
options=options,
|
|
606
|
+
):
|
|
607
|
+
... # handle results
|
|
608
|
+
except grpc.RpcError as error:
|
|
609
|
+
print(error.code(), error.details())
|
|
610
|
+
```
|
|
611
|
+
|
|
612
|
+
## Use with AI Assistants (MCP)
|
|
613
|
+
|
|
614
|
+
The SDK includes an [MCP](https://modelcontextprotocol.io/) server (available since version 0.7.0), so AI assistants such as Claude Code, Codex, Claude Desktop, and Cursor can analyze audio and video files for you.
|
|
560
615
|
|
|
561
|
-
The SDK includes an [MCP](https://modelcontextprotocol.io/) server (available since version 0.7.0), so AI assistants such as Claude Code, Codex, Claude Desktop and Cursor can analyze audio and video files for you.
|
|
562
616
|
The setups below start the server with `uvx`, so install [uv](https://docs.astral.sh/uv/) first.
|
|
563
617
|
|
|
564
|
-
The server reads
|
|
565
|
-
|
|
618
|
+
The server reads credentials from `BEHAVIORALSIGNALS_CID` and `BEHAVIORALSIGNALS_API_KEY`. Your assistant starts the server, so provide them in the assistant's MCP configuration as shown below.
|
|
619
|
+
|
|
620
|
+
### Claude Code
|
|
566
621
|
|
|
567
|
-
|
|
622
|
+
With the two variables set in your shell, run:
|
|
568
623
|
|
|
569
624
|
```bash
|
|
570
625
|
claude mcp add --env BEHAVIORALSIGNALS_CID="$BEHAVIORALSIGNALS_CID" \
|
|
@@ -572,9 +627,11 @@ claude mcp add --env BEHAVIORALSIGNALS_CID="$BEHAVIORALSIGNALS_CID" \
|
|
|
572
627
|
behavioralsignals -- uvx --python ">=3.10" --from "behavioralsignals[mcp]" behavioralsignals-mcp
|
|
573
628
|
```
|
|
574
629
|
|
|
575
|
-
This adds the server
|
|
630
|
+
This adds the server to the current project only. Your key stays out of your shell history and out of the repository.
|
|
576
631
|
|
|
577
|
-
|
|
632
|
+
### Codex
|
|
633
|
+
|
|
634
|
+
With the two variables set in your shell, run:
|
|
578
635
|
|
|
579
636
|
```bash
|
|
580
637
|
codex mcp add behavioralsignals --env BEHAVIORALSIGNALS_CID="$BEHAVIORALSIGNALS_CID" \
|
|
@@ -582,16 +639,24 @@ codex mcp add behavioralsignals --env BEHAVIORALSIGNALS_CID="$BEHAVIORALSIGNALS_
|
|
|
582
639
|
-- uvx --python ">=3.10" --from "behavioralsignals[mcp]" behavioralsignals-mcp
|
|
583
640
|
```
|
|
584
641
|
|
|
585
|
-
This adds the server for all your projects
|
|
642
|
+
This adds the server for all your projects in `~/.codex/config.toml`.
|
|
643
|
+
|
|
644
|
+
### Claude Desktop and Cursor
|
|
586
645
|
|
|
587
|
-
|
|
646
|
+
Add this to `claude_desktop_config.json` (Claude Desktop: **Settings > Developer > Edit Config**) or `~/.cursor/mcp.json`:
|
|
588
647
|
|
|
589
648
|
```json
|
|
590
649
|
{
|
|
591
650
|
"mcpServers": {
|
|
592
651
|
"behavioralsignals": {
|
|
593
652
|
"command": "uvx",
|
|
594
|
-
"args": [
|
|
653
|
+
"args": [
|
|
654
|
+
"--python",
|
|
655
|
+
">=3.10",
|
|
656
|
+
"--from",
|
|
657
|
+
"behavioralsignals[mcp]",
|
|
658
|
+
"behavioralsignals-mcp"
|
|
659
|
+
],
|
|
595
660
|
"env": {
|
|
596
661
|
"BEHAVIORALSIGNALS_CID": "your_cid",
|
|
597
662
|
"BEHAVIORALSIGNALS_API_KEY": "your_api_key"
|
|
@@ -601,32 +666,63 @@ This adds the server for all your projects, in `~/.codex/config.toml`.
|
|
|
601
666
|
}
|
|
602
667
|
```
|
|
603
668
|
|
|
604
|
-
If the app cannot find `uvx`, use its full path (
|
|
669
|
+
If the app cannot find `uvx`, use its full path (`which uvx`).
|
|
670
|
+
|
|
671
|
+
### MCP Tools
|
|
605
672
|
|
|
606
|
-
The server
|
|
673
|
+
The server exposes four tools:
|
|
607
674
|
|
|
608
675
|
| Tool | What it does |
|
|
609
676
|
|---|---|
|
|
610
677
|
| `analyze_behavior` | Uploads an audio file or S3 presigned URL for behavioral analysis and returns the results |
|
|
611
|
-
| `detect_deepfake` | Uploads an audio or video file, or an S3 presigned URL, for deepfake detection and returns the results |
|
|
612
|
-
| `get_result` | Returns the results of a process, in pages, optionally
|
|
613
|
-
| `list_processes` | Lists
|
|
678
|
+
| `detect_deepfake` | Uploads an audio or video file, or an S3 presigned URL, for deepfake detection (video is experimental) and returns the results |
|
|
679
|
+
| `get_result` | Returns the results of a process, in pages, optionally filtered to specific tasks. Each row has the final label and the probability of every label |
|
|
680
|
+
| `list_processes` | Lists processes, newest first, including the failure reason when available |
|
|
614
681
|
|
|
615
682
|
Notes:
|
|
616
|
-
* Each upload uses credits, and the files you name are sent to the Behavioral Signals API.
|
|
617
|
-
* The upload tools wait up to `wait_seconds` (45 by default, 50 at most). If processing takes longer, they return the process ID, and the assistant checks it later with `get_result`.
|
|
618
|
-
* Don't commit config files that contain your API key.
|
|
619
683
|
|
|
620
|
-
|
|
684
|
+
- Each upload uses API credits, and files named in MCP tool calls are sent to the Behavioral Signals API.
|
|
685
|
+
- Upload tools wait up to `wait_seconds` (45 seconds by default, 50 seconds at most). If processing takes longer, they return the process ID so the assistant can check it later with `get_result`.
|
|
686
|
+
- Do not commit configuration files containing your API key.
|
|
621
687
|
|
|
622
|
-
|
|
688
|
+
### Running the MCP Server Manually
|
|
689
|
+
|
|
690
|
+
With `uv` installed, run the server without installing the package:
|
|
623
691
|
|
|
624
692
|
```bash
|
|
625
693
|
uvx --python ">=3.10" --from "behavioralsignals[mcp]" behavioralsignals-mcp
|
|
626
694
|
```
|
|
627
695
|
|
|
628
|
-
Or install
|
|
696
|
+
Or install the MCP extra with pip:
|
|
629
697
|
|
|
630
698
|
```bash
|
|
631
699
|
pip install "behavioralsignals[mcp]"
|
|
700
|
+
behavioralsignals-mcp
|
|
632
701
|
```
|
|
702
|
+
|
|
703
|
+
## Requirements
|
|
704
|
+
|
|
705
|
+
- Python 3.10+
|
|
706
|
+
- A Behavioral Signals account and API key
|
|
707
|
+
|
|
708
|
+
`ffmpeg` is needed by `make_audio_stream()` for formats other than WAV, such as mp3.
|
|
709
|
+
|
|
710
|
+
## Documentation and Examples
|
|
711
|
+
|
|
712
|
+
- [Python SDK Documentation](https://behavioralsignals.readme.io/docs/behavioral-signals-python-sdk)
|
|
713
|
+
- [Full API Documentation](https://behavioralsignals.readme.io/)
|
|
714
|
+
- [Streaming with the Python SDK](https://behavioralsignals.readme.io/docs/streaming-using-python-sdk)
|
|
715
|
+
- [Video Deepfake Detection](https://behavioralsignals.readme.io/docs/submit-a-file-for-processing)
|
|
716
|
+
- [Batch Examples](examples/batch/)
|
|
717
|
+
- [Streaming Examples](examples/streaming/)
|
|
718
|
+
- [AI-friendly Documentation (`llms.txt`)](https://behavioralsignals.readme.io/llms.txt)
|
|
719
|
+
|
|
720
|
+
## Contributing
|
|
721
|
+
|
|
722
|
+
Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, testing, formatting, and pull request guidelines.
|
|
723
|
+
|
|
724
|
+
Please report security issues according to [SECURITY.md](SECURITY.md).
|
|
725
|
+
|
|
726
|
+
## License
|
|
727
|
+
|
|
728
|
+
Licensed under the Apache License 2.0. See [LICENSE](LICENSE).
|