interhumanai 3.0.0__tar.gz → 3.2.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -5,6 +5,36 @@ All notable changes to `interhumanai` are documented here. The package follows
5
5
  the TypeScript SDK (`@interhumanai/sdk`): both packages always carry the same
6
6
  version and release together.
7
7
 
8
+ ## 3.2.1
9
+
10
+ - Documentation only: when no speaker could be identified as on camera, the
11
+ lexical and speech cues of an ``UploadModel.INTER_2_DEEP`` job cover every
12
+ speaker, with ``speaker_role`` as ``None``, rather than the block being
13
+ ``FAILED``.
14
+
15
+ ## 3.2.0
16
+
17
+ - Added three ``UploadJobIncludeFlag`` values for ``UploadModel.INTER_2_DEEP``
18
+ jobs, with their result types: ``VISUAL_CUES``
19
+ (``UploadJobResult.visual_cues``, behaviors seen on camera), ``LEXICAL_CUES``
20
+ (``UploadJobResult.lexical_cues``, features of how the words were chosen) and
21
+ ``SPEECH_CUES`` (``UploadJobResult.speech_cues``, changes in how a speaker
22
+ sounds). Each block carries its own ``status``, so cues that could not be
23
+ produced do not fail the job. These sections are defined ahead of being
24
+ served: until a deployment serves one, a submit that names its flag is
25
+ refused with ``ih1003``.
26
+
27
+ ## 3.1.0
28
+
29
+ - Added ``UploadJobIncludeFlag.TRANSCRIPT`` and ``UploadJobResult.transcript``.
30
+ A ``UploadModel.INTER_2_DEEP`` job submitted with the flag carries a
31
+ transcript of the whole file: every word with its times and speaker
32
+ (``UploadJobTranscriptWord``), and who spoke when
33
+ (``UploadJobTranscriptSegment``). ``UploadJobTranscript.status`` tells a
34
+ transcribed file, even one with no speech, from a transcript that could not
35
+ be produced; the second carries ``error`` and does not fail the job. The API
36
+ refuses the flag for any other model.
37
+
8
38
  ## 3.0.0
9
39
 
10
40
  - **Breaking:** renamed the five section fields of ``UploadJobReasoning`` to
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: interhumanai
3
- Version: 3.0.0
3
+ Version: 3.2.1
4
4
  Summary: First-party Python SDK for the Interhuman API: upload and stream social-signal analysis.
5
5
  Project-URL: Homepage, https://docs.interhuman.ai
6
6
  Project-URL: Repository, https://github.com/interhuman-core/interhuman-api
@@ -167,7 +167,16 @@ else:
167
167
  named. `SIGNALS` adds `signals`: every window's signals as one chronological
168
168
  list, same-type signals in adjacent windows merged into one span with the
169
169
  highest probability and its rationale, and no `modality` (an empty list when
170
- nothing was detected). Without a flag the result has no such section.
170
+ nothing was detected). `TRANSCRIPT`, which `UploadModel.INTER_2_DEEP` jobs
171
+ accept, adds `transcript`: an `UploadJobTranscript` with every spoken word
172
+ (`text`, `start`, `end`, `speaker`, `speaker_role`) and the speaker
173
+ segments. Check `transcript.status` first: `FAILED` means no transcript
174
+ could be produced, carries `error`, and leaves the job completed.
175
+ `VISUAL_CUES`, `LEXICAL_CUES` and `SPEECH_CUES` add `visual_cues`,
176
+ `lexical_cues` and `speech_cues`: timestamped cue lists, each with its own
177
+ `status`. The API defines them ahead of serving them, so a submit that names
178
+ one is refused with `ih1003` until the deployment serves it. Without a flag
179
+ the result has no such section.
171
180
  - `get_job(job_id)` — one status read. Raises `InterhumanAPIError` with
172
181
  `error_id` `ih4021` once the job has expired (`job.expires_at`).
173
182
  - `wait_for_job(job_id, timeout=None, poll_interval=2.0)` — returns the
@@ -141,7 +141,16 @@ else:
141
141
  named. `SIGNALS` adds `signals`: every window's signals as one chronological
142
142
  list, same-type signals in adjacent windows merged into one span with the
143
143
  highest probability and its rationale, and no `modality` (an empty list when
144
- nothing was detected). Without a flag the result has no such section.
144
+ nothing was detected). `TRANSCRIPT`, which `UploadModel.INTER_2_DEEP` jobs
145
+ accept, adds `transcript`: an `UploadJobTranscript` with every spoken word
146
+ (`text`, `start`, `end`, `speaker`, `speaker_role`) and the speaker
147
+ segments. Check `transcript.status` first: `FAILED` means no transcript
148
+ could be produced, carries `error`, and leaves the job completed.
149
+ `VISUAL_CUES`, `LEXICAL_CUES` and `SPEECH_CUES` add `visual_cues`,
150
+ `lexical_cues` and `speech_cues`: timestamped cue lists, each with its own
151
+ `status`. The API defines them ahead of serving them, so a submit that names
152
+ one is refused with `ih1003` until the deployment serves it. Without a flag
153
+ the result has no such section.
145
154
  - `get_job(job_id)` — one status read. Raises `InterhumanAPIError` with
146
155
  `error_id` `ih4021` once the job has expired (`job.expires_at`).
147
156
  - `wait_for_job(job_id, timeout=None, poll_interval=2.0)` — returns the
@@ -124,12 +124,23 @@ from .types import (
124
124
  StreamModel,
125
125
  UploadJob,
126
126
  UploadJobConversationQuality,
127
+ UploadJobCuesStatus,
127
128
  UploadJobError,
128
129
  UploadJobIncludeFlag,
130
+ UploadJobLexicalCue,
131
+ UploadJobLexicalCues,
129
132
  UploadJobReasoning,
130
133
  UploadJobResult,
131
134
  UploadJobSignal,
135
+ UploadJobSpeechCue,
136
+ UploadJobSpeechCues,
132
137
  UploadJobStatus,
138
+ UploadJobTranscript,
139
+ UploadJobTranscriptSegment,
140
+ UploadJobTranscriptStatus,
141
+ UploadJobTranscriptWord,
142
+ UploadJobVisualCue,
143
+ UploadJobVisualCues,
133
144
  UploadJobWindow,
134
145
  UploadModel,
135
146
  )
@@ -227,13 +238,24 @@ __all__ = [
227
238
  "UploadClient",
228
239
  "UploadJob",
229
240
  "UploadJobConversationQuality",
241
+ "UploadJobCuesStatus",
230
242
  "UploadJobError",
231
243
  "UploadJobIncludeFlag",
244
+ "UploadJobLexicalCue",
245
+ "UploadJobLexicalCues",
232
246
  "UploadJobReasoning",
233
247
  "UploadJobResult",
234
248
  "UploadJobSignal",
249
+ "UploadJobSpeechCue",
250
+ "UploadJobSpeechCues",
235
251
  "UploadJobStatus",
236
252
  "UploadJobTimeoutError",
253
+ "UploadJobTranscript",
254
+ "UploadJobTranscriptSegment",
255
+ "UploadJobTranscriptStatus",
256
+ "UploadJobTranscriptWord",
257
+ "UploadJobVisualCue",
258
+ "UploadJobVisualCues",
237
259
  "UploadJobWindow",
238
260
  "UploadModel",
239
261
  "VideoInput",
@@ -1,3 +1,3 @@
1
1
  """Package version (single source of truth, read by hatchling at build time)."""
2
2
 
3
- __version__ = "3.0.0"
3
+ __version__ = "3.2.1"
@@ -250,13 +250,26 @@ class UploadJobIncludeFlag(str, enum.Enum):
250
250
 
251
251
  The two :class:`IncludeFlag` Conversation Quality Index sections, plus
252
252
  ``SIGNALS`` for the whole-file signal list
253
- (:attr:`UploadJobResult.signals`). ``SIGNALS`` is a v2 upload option only;
254
- :meth:`~interhumanai.UploadClient.analyze` and the stream do not take it.
253
+ (:attr:`UploadJobResult.signals`) and ``TRANSCRIPT`` for a transcript of
254
+ the whole file (:attr:`UploadJobResult.transcript`). Both are v2 upload
255
+ options only; :meth:`~interhumanai.UploadClient.analyze` and the stream do
256
+ not take them. ``TRANSCRIPT`` is accepted with ``UploadModel.INTER_2_DEEP``
257
+ only; the API refuses it for any other model.
258
+
259
+ ``VISUAL_CUES``, ``LEXICAL_CUES`` and ``SPEECH_CUES`` ask for timestamped
260
+ cue lists (:attr:`UploadJobResult.visual_cues`,
261
+ :attr:`UploadJobResult.lexical_cues`, :attr:`UploadJobResult.speech_cues`).
262
+ They are ``UploadModel.INTER_2_DEEP`` options too. A deployment that does
263
+ not serve a section yet refuses a submit that names it with ``ih1003``.
255
264
  """
256
265
 
257
266
  CONVERSATION_QUALITY_OVERALL = "conversation_quality_overall"
258
267
  CONVERSATION_QUALITY_TIMELINE = "conversation_quality_timeline"
259
268
  SIGNALS = "signals"
269
+ TRANSCRIPT = "transcript"
270
+ VISUAL_CUES = "visual_cues"
271
+ LEXICAL_CUES = "lexical_cues"
272
+ SPEECH_CUES = "speech_cues"
260
273
 
261
274
 
262
275
  class UploadJobStatus(str, enum.Enum):
@@ -358,6 +371,284 @@ class UploadJobConversationQuality(BaseModel):
358
371
  timeline: list[ConversationQualityTimelineEntry] | None = None
359
372
 
360
373
 
374
+ class UploadJobError(BaseModel):
375
+ """Why an upload job failed, in the API's standard error-body shape."""
376
+
377
+ error_id: str
378
+ correlation_id: str | None = None
379
+ link: str | None = None
380
+ message: str | None = None
381
+
382
+
383
+ class UploadJobTranscriptStatus(str, enum.Enum):
384
+ """How the transcript of an upload job ended.
385
+
386
+ ``COMPLETED`` means the file was transcribed, including when nobody speaks
387
+ in it and the transcript is empty. ``FAILED`` means no transcript could be
388
+ produced; the job's windows are unaffected.
389
+ """
390
+
391
+ COMPLETED = "completed"
392
+ FAILED = "failed"
393
+
394
+
395
+ class UploadJobTranscriptWord(BaseModel):
396
+ """One spoken word of an upload job's file.
397
+
398
+ Attributes:
399
+ text: The word as it was spoken. The transcript is verbatim: fillers
400
+ and repeated words are kept.
401
+ start: Where the word starts, in seconds from the start of the file.
402
+ end: Where the word ends, in seconds from the start of the file.
403
+ speaker: The speaker who said the word, as the number the transcript's
404
+ segments use. ``None`` when the word falls outside every speaker
405
+ segment.
406
+ speaker_role: ``"fg_speaker"`` for the person on camera,
407
+ ``"bg_speaker"`` for a voice from off camera, or ``None`` when the
408
+ speaker's role could not be told.
409
+ """
410
+
411
+ text: str
412
+ start: float
413
+ end: float
414
+ speaker: int | None = None
415
+ speaker_role: Literal["fg_speaker", "bg_speaker"] | None = None
416
+
417
+
418
+ class UploadJobTranscriptSegment(BaseModel):
419
+ """One stretch of speech by one speaker.
420
+
421
+ Attributes:
422
+ speaker: The speaker's number, unique within the file and starting at 0.
423
+ start: Where the segment starts, in seconds from the start of the file.
424
+ end: Where the segment ends, in seconds from the start of the file.
425
+ speaker_role: ``"fg_speaker"`` for the person on camera,
426
+ ``"bg_speaker"`` for a voice from off camera, or ``None`` when the
427
+ speaker's role could not be told.
428
+ """
429
+
430
+ speaker: int
431
+ start: float
432
+ end: float
433
+ speaker_role: Literal["fg_speaker", "bg_speaker"] | None = None
434
+
435
+
436
+ class UploadJobTranscript(BaseModel):
437
+ """The transcript of a completed ``UploadModel.INTER_2_DEEP`` job that asked for one.
438
+
439
+ Read ``status`` first. ``COMPLETED`` carries ``words`` and ``segments``,
440
+ both empty when nobody speaks in the file. ``FAILED`` carries ``error`` and
441
+ no words: the transcript could not be produced, which is a different
442
+ outcome from a file without speech. Either way the job itself completed
443
+ and its windows are whole.
444
+
445
+ Attributes:
446
+ status: Whether the file was transcribed.
447
+ language: The language detected in the speech, as an ISO 639 code.
448
+ speaker_roles_status: Whether the speakers could be told apart as on
449
+ camera or off it: ``"mapped"`` when at least one speaker has a
450
+ role, ``"unmapped"`` when none could be told, ``"no_face"`` when no
451
+ face was found in the picture, ``"failed"`` when the step itself
452
+ failed. Only ``speaker_role`` is affected; the words and segments
453
+ are complete in every case. Treat a value not listed here as
454
+ ``"unmapped"``: later versions of the API may add one.
455
+ word_timing: How the word times were measured. ``"asr"`` means they
456
+ are the speech recognizer's own estimates.
457
+ words: Every spoken word, in the order spoken.
458
+ segments: Who spoke when: one entry per stretch of speech, in order.
459
+ error: Why no transcript could be produced. Set only when ``status``
460
+ is ``FAILED``.
461
+ """
462
+
463
+ status: UploadJobTranscriptStatus
464
+ language: str | None = None
465
+ speaker_roles_status: str | None = None
466
+ word_timing: str | None = None
467
+ words: list[UploadJobTranscriptWord] = Field(default_factory=list)
468
+ segments: list[UploadJobTranscriptSegment] = Field(default_factory=list)
469
+ error: UploadJobError | None = None
470
+
471
+
472
+ class UploadJobCuesStatus(str, enum.Enum):
473
+ """How one family of cues of an upload job ended.
474
+
475
+ ``COMPLETED`` means the stage ran over the whole file, including when it
476
+ found no cue and the list is empty. ``FAILED`` means the stage could not
477
+ produce its cues; the job's windows and its other sections are unaffected.
478
+ """
479
+
480
+ COMPLETED = "completed"
481
+ FAILED = "failed"
482
+
483
+
484
+ class UploadJobVisualCue(BaseModel):
485
+ """One observable behavior of a person on camera.
486
+
487
+ Attributes:
488
+ label: A short name for the behavior, in lower case. The vocabulary is
489
+ open: the same movement can arrive under more than one name.
490
+ start: Where the behavior starts, in seconds from the start of the file.
491
+ end: Where the behavior ends, in seconds from the start of the file.
492
+ description: One sentence describing what was seen.
493
+ visible_participant: Which person on camera the behavior belongs to,
494
+ as ``"P1"``, ``"P2"`` and so on. It names a body in the picture,
495
+ which is not the transcript's ``speaker`` number.
496
+ context_speaker: Who was speaking while the behavior happened:
497
+ ``"fg_speaker"`` for the person on camera, ``"bg_speaker"`` for a
498
+ voice from off camera, or ``None`` when nobody was speaking or the
499
+ role could not be told.
500
+ """
501
+
502
+ label: str
503
+ start: float
504
+ end: float
505
+ description: str | None = None
506
+ visible_participant: str | None = None
507
+ context_speaker: Literal["fg_speaker", "bg_speaker"] | None = None
508
+
509
+
510
+ class UploadJobLexicalCue(BaseModel):
511
+ """One feature of how a speaker chose and built their words.
512
+
513
+ Attributes:
514
+ label: A short name for what was said, in lower case, such as
515
+ ``"repetition"``, ``"false start"``, ``"hedge"`` or
516
+ ``"filled pause"``. The vocabulary is open: it covers
517
+ disfluencies and also hedges, intensifiers, vague language,
518
+ discourse markers, questions and laughter.
519
+ start: Where the cue starts, in seconds from the start of the file.
520
+ end: Where the cue ends, in seconds from the start of the file.
521
+ quote: The words the cue covers, as spoken.
522
+ core_quote: The fragment the label names, inside ``quote``.
523
+ description: One sentence describing the cue.
524
+ speaker: The speaker, as the number the transcript's segments use.
525
+ speaker_role: ``"fg_speaker"`` for the person on camera,
526
+ ``"bg_speaker"`` for a voice from off camera, or ``None`` when the
527
+ role could not be told.
528
+ word_start: Position of the cue's first word in the
529
+ ``transcript.words`` list of this same result, counting from 0.
530
+ ``None`` when the result carries no completed transcript.
531
+ word_end: Position of the cue's last word in that list, inclusive.
532
+ ``None`` under the same condition.
533
+ """
534
+
535
+ label: str
536
+ start: float
537
+ end: float
538
+ quote: str
539
+ core_quote: str | None = None
540
+ description: str | None = None
541
+ speaker: int | None = None
542
+ speaker_role: Literal["fg_speaker", "bg_speaker"] | None = None
543
+ word_start: int | None = None
544
+ word_end: int | None = None
545
+
546
+
547
+ class UploadJobSpeechCue(BaseModel):
548
+ """One change in how a speaker sounds, or one turn-taking event.
549
+
550
+ Attributes:
551
+ label: What was heard, in lower case, such as ``"faster than usual"``,
552
+ ``"higher pitch than usual"`` or ``"backchannel"``.
553
+ feature: What was measured: ``"speech_rate"``, ``"pitch_level"``,
554
+ ``"pitch_range"``, ``"pitch_slope"``, ``"loudness"``,
555
+ ``"response_latency"``, ``"reply_length"``,
556
+ ``"disfluency_rate"``, ``"interruption"`` or ``"backchannel"``.
557
+ Treat a value not listed here as unknown: later versions of the
558
+ API may add one.
559
+ start: Where the cue starts, in seconds from the start of the file.
560
+ end: Where the cue ends, in seconds from the start of the file.
561
+ value: The measurement, in the unit of its ``feature``: syllables
562
+ per second for ``speech_rate``; semitones (12 times the base-2
563
+ logarithm of the frequency in hertz) for ``pitch_level``;
564
+ semitones between the low and the high of the pitch for
565
+ ``pitch_range``; semitones per second for ``pitch_slope``;
566
+ decibels for ``loudness``; seconds before the reply for
567
+ ``response_latency``; words for ``reply_length``; hesitations per
568
+ 100 words for ``disfluency_rate``; seconds of overlap for
569
+ ``interruption``. ``None`` for a ``backchannel``.
570
+ z: How unusual the measurement is, in standard deviations from the
571
+ rest of the analyzed speech in this file. For ``reply_length`` it
572
+ is taken on a logarithmic scale. ``None`` for a ``backchannel``
573
+ or an ``interruption``, which are not compared with a baseline.
574
+ description: One sentence describing the cue.
575
+ quote: The words spoken during the cue, when it covers words.
576
+ speaker: The speaker, as the number the transcript's segments use.
577
+ speaker_role: ``"fg_speaker"`` for the person on camera,
578
+ ``"bg_speaker"`` for a voice from off camera, or ``None`` when the
579
+ role could not be told.
580
+ word_start: Position of the cue's first word in the
581
+ ``transcript.words`` list of this same result, counting from 0.
582
+ ``None`` when the cue covers no words or the result carries no
583
+ completed transcript.
584
+ word_end: Position of the cue's last word in that list, inclusive.
585
+ ``None`` under the same conditions.
586
+ """
587
+
588
+ label: str
589
+ feature: str
590
+ start: float
591
+ end: float
592
+ value: float | None = None
593
+ z: float | None = None
594
+ description: str | None = None
595
+ quote: str | None = None
596
+ speaker: int | None = None
597
+ speaker_role: Literal["fg_speaker", "bg_speaker"] | None = None
598
+ word_start: int | None = None
599
+ word_end: int | None = None
600
+
601
+
602
+ class UploadJobVisualCues(BaseModel):
603
+ """The visual cues of a completed ``UploadModel.INTER_2_DEEP`` job that asked for them.
604
+
605
+ Read ``status`` first. ``COMPLETED`` carries ``cues``, empty when nothing
606
+ was seen. ``FAILED`` carries ``error`` and no cues. Either way the job
607
+ itself completed.
608
+ """
609
+
610
+ status: UploadJobCuesStatus
611
+ cues: list[UploadJobVisualCue] = Field(default_factory=list)
612
+ error: UploadJobError | None = None
613
+
614
+
615
+ class UploadJobLexicalCues(BaseModel):
616
+ """The lexical cues of a completed ``UploadModel.INTER_2_DEEP`` job that asked for them.
617
+
618
+ Read ``status`` first. ``COMPLETED`` carries ``cues``, empty when nothing
619
+ stood out. ``FAILED`` carries ``error`` and no cues. Either way the job
620
+ itself completed.
621
+
622
+ The cues cover the speech of the person on camera. An off-camera speaker
623
+ is not analyzed. When no speaker could be identified as on camera, every
624
+ speaker is analyzed and each cue's ``speaker_role`` is ``None``.
625
+ """
626
+
627
+ status: UploadJobCuesStatus
628
+ cues: list[UploadJobLexicalCue] = Field(default_factory=list)
629
+ error: UploadJobError | None = None
630
+
631
+
632
+ class UploadJobSpeechCues(BaseModel):
633
+ """The speech cues of a completed ``UploadModel.INTER_2_DEEP`` job that asked for them.
634
+
635
+ Read ``status`` first. ``COMPLETED`` carries ``cues``, empty when nothing
636
+ stood out or the file holds too little speech to judge a speaker against
637
+ themselves. ``FAILED`` carries ``error`` and no cues. Either way the job
638
+ itself completed.
639
+
640
+ The cues cover the speech of the person on camera, judged against the rest
641
+ of that person's speech in the file. An off-camera speaker is not
642
+ analyzed. When no speaker could be identified as on camera, every speaker
643
+ is analyzed, each against the rest of the speech in the file, and each
644
+ cue's ``speaker_role`` is ``None``.
645
+ """
646
+
647
+ status: UploadJobCuesStatus
648
+ cues: list[UploadJobSpeechCue] = Field(default_factory=list)
649
+ error: UploadJobError | None = None
650
+
651
+
361
652
  class UploadJobResult(BaseModel):
362
653
  """The result of a completed upload job: one entry per analyzed window.
363
654
 
@@ -371,6 +662,17 @@ class UploadJobResult(BaseModel):
371
662
 
372
663
  ``conversation_quality`` is present only when the submit passed at least one
373
664
  Conversation Quality Index flag, and carries the sections those flags named.
665
+
666
+ ``transcript`` is a transcript of the whole file with speakers and word
667
+ times. It is set only when the submit passed
668
+ :attr:`UploadJobIncludeFlag.TRANSCRIPT`, which ``UploadModel.INTER_2_DEEP``
669
+ jobs accept. A transcript that could not be produced is still set, with
670
+ ``status`` ``FAILED``, and does not fail the job.
671
+
672
+ ``visual_cues``, ``lexical_cues`` and ``speech_cues`` are timestamped cue
673
+ lists, each set only when the submit passed the matching
674
+ :class:`UploadJobIncludeFlag`. Each carries its own ``status``, so one
675
+ family that could not be produced does not fail the job or the others.
374
676
  """
375
677
 
376
678
  duration_seconds: float
@@ -378,15 +680,10 @@ class UploadJobResult(BaseModel):
378
680
  windows: list[UploadJobWindow] = Field(default_factory=list)
379
681
  signals: list[UploadJobSignal] | None = None
380
682
  conversation_quality: UploadJobConversationQuality | None = None
381
-
382
-
383
- class UploadJobError(BaseModel):
384
- """Why an upload job failed, in the API's standard error-body shape."""
385
-
386
- error_id: str
387
- correlation_id: str | None = None
388
- link: str | None = None
389
- message: str | None = None
683
+ transcript: UploadJobTranscript | None = None
684
+ visual_cues: UploadJobVisualCues | None = None
685
+ lexical_cues: UploadJobLexicalCues | None = None
686
+ speech_cues: UploadJobSpeechCues | None = None
390
687
 
391
688
 
392
689
  class UploadJob(BaseModel):
File without changes
File without changes