livekit-plugins-duplexjev 0.2.0__tar.gz → 0.3.1__tar.gz

{livekit_plugins_duplexjev-0.2.0 → livekit_plugins_duplexjev-0.3.1}/CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## 0.3.1 (2026-10-13)
+
+- Logs a one-time hint when the free trial API is used: it runs DuplexJev-4B-Para, so
+  its turn-taking does not match the published eot-bench numbers (DuplexJev-32B-Para v1.2 behind your own
+  `duplexjev gateway` >= 0.6).
+- README: how to reproduce the eot-bench numbers; the self-hosting example now serves DuplexJev-32B-Para v1.2 with
+  `--rate 1000000`; eot-bench figure for the default setting updated to the full run (20.0%).
+
+## 0.3.0 (2026-10-13)
+
+- `readout="act"`: ask "What should the assistant do now?" and report P(reply now) + P(short acknowledgement) as the
+  end-of-turn probability (also gives the backchannel probability from the same answer). On LiveKit eot-bench
+  (English, 32B-Para v1.2, with the agent's previous message) false cut-offs at 300 ms: 18.4% vs 19.7%.
+- `last_turn_class` (finished / not_finished / backchannel / hesitating, or reply / keep_listening / acknowledge).
+
 ## 0.2.0 (2026-10-13)
 
 - Turn-taking context: `detector.attach(session)` (or `detector.set_assistant_said(text)`) sends the agent's previous

{livekit_plugins_duplexjev-0.2.0 → livekit_plugins_duplexjev-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: livekit-plugins-duplexjev
-Version: 0.2.0
+Version: 0.3.1
 Summary: Audio end-of-turn detection for LiveKit Agents with DuplexJev: turn state, emotion and non-verbal sounds from one request, no transcript needed.
 Project-URL: Homepage, https://github.com/adventists-ai/duplexjev/tree/main/integrations/livekit
 Project-URL: Models, https://huggingface.co/adventists-ai/DuplexJev-4B-Para
@@ -43,6 +43,12 @@ say, and tells your agent whether the user is **finished**, **still talking**, *
 
 Tested with `livekit-agents` 1.8.6. Maintained by Adventists.ai (AI降临派).
 
+> **Getting the published turn-taking numbers.** LiveKit eot-bench (English, false cut-offs at 300 ms): **20.0%** for
+> DuplexJev-32B-Para v1.2 with the agent's previous message, 23.5% audio only, vs 35.2% for Smart Turn v3.2. That needs
+> DuplexJev-32B-Para **v1.2 on your own GPU** behind `duplexjev gateway` >= 0.6 (see *Run the model yourself*), the
+> default `readout="turn"`, and the agent's previous message (below). The free trial API runs the smaller DuplexJev-4B-Para
+> (no v1.2 turn-taking update): use it to try the plugin, not to judge turn-taking quality.
+
 ## Install
 
 ```bash
@@ -75,8 +81,8 @@ session = AgentSession(..., turn_handling={"turn_detection": detector})
 detector.attach(session)        # follows conversation_item_added; or detector.set_assistant_said(text)
 ```
 
-It needs a gateway from `duplexjev>=0.6`. On LiveKit eot-bench (English, DuplexJev-32B-Para v1.2) this cuts false
-cut-offs at a 300 ms budget from 23.5% to 19.7%.
+It needs a gateway from `duplexjev>=0.6`. On LiveKit eot-bench (English, DuplexJev-32B-Para v1.2, full run) this cuts
+false cut-offs at a 300 ms budget from 23.5% to 20.0%.
 
 `AgentSession(turn_detection=duplexjev.TurnDetector(), ...)` works too (LiveKit marks that argument as deprecated).
 A complete agent is in [`examples/agent.py`](examples/agent.py).
@@ -92,14 +98,18 @@ commits the turn, as with any turn detector, because the LLM needs the text.
 
 ## Run the model yourself (recommended for real-time)
 
-The default URL is our free trial API (rate limited to 30 requests per minute; the round trip from outside mainland China adds network latency). For a live agent, serve the model next to it (DuplexJev-4B-Para needs about 10 GB of GPU memory; for the best turn-taking use DuplexJev-32B-Para v1.2 on one 80 GB GPU):
+The default URL is our free trial API (rate limited to 30 requests per minute; the round trip from outside mainland China adds network latency; it runs the smaller DuplexJev-4B-Para without the v1.2 turn-taking update). For a live agent, serve the model next to it. For turn-taking, use DuplexJev-32B-Para v1.2 on one 80 GB GPU (the eot-bench numbers above):
 
 ```bash
 pip install "vllm[audio]>=0.29" duplexjev-vllm "duplexjev[server]>=0.6"
-vllm serve adventists-ai/DuplexJev-4B-Para --max-model-len 4096 --port 8000 &
-duplexjev gateway --vllm http://localhost:8000/v1 --port 8420
+vllm serve adventists-ai/DuplexJev-32B-Para --revision v1.2 --max-model-len 4096 --port 8000 &
+duplexjev gateway --vllm http://localhost:8000/v1 --port 8420 --rate 1000000
 ```
 
+On a smaller GPU, `adventists-ai/DuplexJev-4B-Para` needs about 10 GB; it is fine for emotion and the other questions,
+but its turn-taking has not had the v1.2 update. `--rate 1000000` lifts the gateway's default limit of 30 requests per
+minute, which a live agent exceeds quickly.
+
 Then pass `api_url="http://localhost:8420"` (or set `DUPLEXJEV_API_URL`).
 
 ## Options
@@ -117,6 +127,10 @@ Then pass `api_url="http://localhost:8420"` (or set `DUPLEXJEV_API_URL`).
 | `trim_silence` | `True` | drop the silence before the user started talking (keeps 0.3 s) |
 | `prediction_timeout` | `4.0` hosted, `1.0` own gateway | how long LiveKit waits for a prediction |
 | `fallback_probability` | `1.0` | P(finished) reported if the gateway fails, so the conversation never blocks |
+| `readout` | `"turn"` | `"act"` asks "What should the assistant do now?" (P(reply) + P(short acknowledgement)); slightly better on eot-bench with context |
+
+After each prediction `detector.last_turn_class` is `finished`, `not_finished`, `backchannel` or `hesitating`
+(`reply`, `keep_listening`, `acknowledge` with `readout="act"`), for your own routing.
 
 ## License
 

{livekit_plugins_duplexjev-0.2.0 → livekit_plugins_duplexjev-0.3.1}/README.md

@@ -17,6 +17,12 @@ say, and tells your agent whether the user is **finished**, **still talking**, *
 
 Tested with `livekit-agents` 1.8.6. Maintained by Adventists.ai (AI降临派).
 
+> **Getting the published turn-taking numbers.** LiveKit eot-bench (English, false cut-offs at 300 ms): **20.0%** for
+> DuplexJev-32B-Para v1.2 with the agent's previous message, 23.5% audio only, vs 35.2% for Smart Turn v3.2. That needs
+> DuplexJev-32B-Para **v1.2 on your own GPU** behind `duplexjev gateway` >= 0.6 (see *Run the model yourself*), the
+> default `readout="turn"`, and the agent's previous message (below). The free trial API runs the smaller DuplexJev-4B-Para
+> (no v1.2 turn-taking update): use it to try the plugin, not to judge turn-taking quality.
+
 ## Install
 
 ```bash
@@ -49,8 +55,8 @@ session = AgentSession(..., turn_handling={"turn_detection": detector})
 detector.attach(session)        # follows conversation_item_added; or detector.set_assistant_said(text)
 ```
 
-It needs a gateway from `duplexjev>=0.6`. On LiveKit eot-bench (English, DuplexJev-32B-Para v1.2) this cuts false
-cut-offs at a 300 ms budget from 23.5% to 19.7%.
+It needs a gateway from `duplexjev>=0.6`. On LiveKit eot-bench (English, DuplexJev-32B-Para v1.2, full run) this cuts
+false cut-offs at a 300 ms budget from 23.5% to 20.0%.
 
 `AgentSession(turn_detection=duplexjev.TurnDetector(), ...)` works too (LiveKit marks that argument as deprecated).
 A complete agent is in [`examples/agent.py`](examples/agent.py).
@@ -66,14 +72,18 @@ commits the turn, as with any turn detector, because the LLM needs the text.
 
 ## Run the model yourself (recommended for real-time)
 
-The default URL is our free trial API (rate limited to 30 requests per minute; the round trip from outside mainland China adds network latency). For a live agent, serve the model next to it (DuplexJev-4B-Para needs about 10 GB of GPU memory; for the best turn-taking use DuplexJev-32B-Para v1.2 on one 80 GB GPU):
+The default URL is our free trial API (rate limited to 30 requests per minute; the round trip from outside mainland China adds network latency; it runs the smaller DuplexJev-4B-Para without the v1.2 turn-taking update). For a live agent, serve the model next to it. For turn-taking, use DuplexJev-32B-Para v1.2 on one 80 GB GPU (the eot-bench numbers above):
 
 ```bash
 pip install "vllm[audio]>=0.29" duplexjev-vllm "duplexjev[server]>=0.6"
-vllm serve adventists-ai/DuplexJev-4B-Para --max-model-len 4096 --port 8000 &
-duplexjev gateway --vllm http://localhost:8000/v1 --port 8420
+vllm serve adventists-ai/DuplexJev-32B-Para --revision v1.2 --max-model-len 4096 --port 8000 &
+duplexjev gateway --vllm http://localhost:8000/v1 --port 8420 --rate 1000000
 ```
 
+On a smaller GPU, `adventists-ai/DuplexJev-4B-Para` needs about 10 GB; it is fine for emotion and the other questions,
+but its turn-taking has not had the v1.2 update. `--rate 1000000` lifts the gateway's default limit of 30 requests per
+minute, which a live agent exceeds quickly.
+
 Then pass `api_url="http://localhost:8420"` (or set `DUPLEXJEV_API_URL`).
 
 ## Options
@@ -91,6 +101,10 @@ Then pass `api_url="http://localhost:8420"` (or set `DUPLEXJEV_API_URL`).
 | `trim_silence` | `True` | drop the silence before the user started talking (keeps 0.3 s) |
 | `prediction_timeout` | `4.0` hosted, `1.0` own gateway | how long LiveKit waits for a prediction |
 | `fallback_probability` | `1.0` | P(finished) reported if the gateway fails, so the conversation never blocks |
+| `readout` | `"turn"` | `"act"` asks "What should the assistant do now?" (P(reply) + P(short acknowledgement)); slightly better on eot-bench with context |
+
+After each prediction `detector.last_turn_class` is `finished`, `not_finished`, `backchannel` or `hesitating`
+(`reply`, `keep_listening`, `acknowledge` with `readout="act"`), for your own routing.
 
 ## License
 

{livekit_plugins_duplexjev-0.2.0 → livekit_plugins_duplexjev-0.3.1}/livekit/plugins/duplexjev/turn_detector.py

@@ -56,6 +56,9 @@ REPLY_QUESTION = {
            "options": ["马上回复", "继续听", "简短回应一下"], "lang": "zh"},
 }
 
+TURN_CLASSES = ["finished", "not_finished", "backchannel", "hesitating"]
+ACT_CLASSES = ["reply", "keep_listening", "acknowledge"]
+
 PRESETS = {
     "emotion": {"en": ("What is the speaker's emotional state?", ["neutral", "happy", "angry", "sad"]),
                 "zh": ("说话人当时的情绪状态是？", ["中性", "高兴", "生气", "伤心"])},
@@ -117,6 +120,10 @@ class TurnDetector:
         extra_questions: names from ``PRESETS`` (``"emotion"``, ``"sound"``, ``"barge_in"``) or question dicts
             ``{"id", "text", "options"}``, answered in the same request; see ``last_answers``.
         on_answers: optional callback ``f(answers: dict)`` called after every successful prediction.
+        readout: ``"turn"`` (turn-state question, default) or ``"act"``: ask "What should the assistant do now?" and
+            report P(reply now) + P(short acknowledgement); slightly better on eot-bench with the agent's previous
+            message (18.4% vs 19.7% false cut-offs at 300 ms, English, 32B-Para v1.2, fast mode; the default
+            ``"turn"`` scores 20.0% in a full eot-bench run).
         window_s: seconds of the current user turn sent with each request (the most recent ones).
         trim_silence: drop the quiet audio before the user started talking, keeping 0.3 s of pre-roll.
         prediction_timeout: how long LiveKit waits for a prediction before committing without one. Defaults to
@@ -142,6 +149,7 @@ class TurnDetector:
         prediction_timeout: float | None = None,
         fallback_probability: float = 1.0,
         http_session: aiohttp.ClientSession | None = None,
+        readout: str = "turn",
     ) -> None:
         if lang not in TURN_QUESTION:
             raise ValueError("lang must be 'en' or 'zh'")
@@ -161,11 +169,19 @@ class TurnDetector:
                 HOSTED_PREDICTION_TIMEOUT if self._base_url == DEFAULT_URL else SELF_HOSTED_PREDICTION_TIMEOUT
             )
         self._prediction_timeout = prediction_timeout
+        if self._base_url == DEFAULT_URL:
+            logger.info("Using the free DuplexJev trial API (DuplexJev-4B-Para, rate limited). "
+                        "For the turn-taking quality reported on eot-bench, serve DuplexJev-32B-Para v1.2 behind "
+                        "`duplexjev gateway` >= 0.6 and pass its URL; see the README.")
         self._fallback_probability = fallback_probability
         self._http_session = http_session
 
-        self._questions: list[dict[str, Any]] = [TURN_QUESTION[lang]]
-        if backchannel_threshold is not None:
+        if readout not in ("turn", "act"):
+            raise ValueError("readout must be 'turn' or 'act'")
+        self._readout = readout
+        main = dict(REPLY_QUESTION[lang], id="turn") if readout == "act" else TURN_QUESTION[lang]
+        self._questions: list[dict[str, Any]] = [main]
+        if backchannel_threshold is not None and readout == "turn":
             self._questions.append(REPLY_QUESTION[lang])
         self._questions += [self._as_question(q) for q in (extra_questions or [])]
         ids = [q["id"] for q in self._questions]
@@ -177,6 +193,9 @@ class TurnDetector:
         """Answers of the latest successful request: ``{id: {"answer", "confidence", "probs"}}``."""
         self.last_server_ms: float | None = None
         """Server-side model time of the latest request, in milliseconds."""
+        self.last_turn_class: str | None = None
+        """Most likely class of the latest request: finished / not_finished / backchannel / hesitating
+        (``readout="act"``: reply / keep_listening / acknowledge)."""
         self.assistant_said: str = ""
         """The agent's previous message, sent with every request (last 300 characters)."""
 
@@ -404,9 +423,12 @@ class TurnDetectorStream:
                 data = await resp.json()
             answers: dict[str, Any] = data["answers"]
             probs = answers["turn"]["probs"]
-            o = TURN_QUESTION[det._lang]["options"]
-            # finished + backchannel: a short "ok" / "yeah" with nobody else talking also ends the turn
+            o = (REPLY_QUESTION if det._readout == "act" else TURN_QUESTION)[det._lang]["options"]
+            # turn: finished + backchannel (a short "ok" with nobody else talking also ends the turn);
+            # act: reply now + short acknowledgement
             p_finished = float(probs.get(o[0], 0.0)) + float(probs.get(o[2], 0.0))
+            names = ACT_CLASSES if det._readout == "act" else TURN_CLASSES
+            det.last_turn_class = names[max(range(len(o)), key=lambda i: float(probs.get(o[i], 0.0)))]
         except asyncio.CancelledError:
             raise
         except Exception as e:  # network or server error: never block the conversation
@@ -425,8 +447,10 @@ class TurnDetectorStream:
         if data.get("model"):
             det._model_name = str(data["model"])
         backchannel = None
-        if "reply" in answers and det._backchannel_threshold is not None:
-            backchannel = float(answers["reply"]["probs"].get(REPLY_QUESTION[det._lang]["options"][2], 0.0))
+        if det._backchannel_threshold is not None:
+            src = answers["turn"] if det._readout == "act" else answers.get("reply")
+            if src is not None:
+                backchannel = float(src["probs"].get(REPLY_QUESTION[det._lang]["options"][2], 0.0))
         now = time.time()
         self.last_answers = det.last_answers = answers
         self.last_server_ms = det.last_server_ms = server_ms

livekit_plugins_duplexjev-0.3.1/livekit/plugins/duplexjev/version.py

@@ -0,0 +1 @@
+__version__ = "0.3.1"

{livekit_plugins_duplexjev-0.2.0 → livekit_plugins_duplexjev-0.3.1}/tests/test_turn_detector.py

@@ -205,3 +205,26 @@ async def test_assistant_said_is_sent_and_attach_follows_session(gateway):
     det.set_assistant_said("y" * 500)
     assert len(det.assistant_said) == 300
     await stream.aclose()
+
+
+async def test_act_readout_and_turn_class(gateway):
+    det = duplexjev.TurnDetector(api_url=gateway.url, readout="act", backchannel_threshold=0.5)
+    stream = det.stream()
+    push(stream, 1.0)
+    gateway.p_finished = 0.8
+    ev = await asyncio.wait_for(stream.predict(), 2)
+    qs = gateway.requests[-1]["questions"]
+    assert [q["id"] for q in qs] == ["turn"] and qs[0]["text"] == "What should the assistant do now?"
+    assert ev.end_of_turn_probability == pytest.approx(0.8) and det.last_turn_class == "reply"
+    assert ev.backchannel_probability == pytest.approx(0.0)
+    gateway.p_finished = 0.2
+    ev = await asyncio.wait_for(stream.predict(), 2)
+    assert det.last_turn_class == "keep_listening"
+    await stream.aclose()
+    t = duplexjev.TurnDetector(api_url=gateway.url)
+    s = t.stream(); push(s, 1.0); gateway.p_finished = 0.9
+    await asyncio.wait_for(s.predict(), 2)
+    assert t.last_turn_class == "finished"
+    await s.aclose()
+    with pytest.raises(ValueError):
+        duplexjev.TurnDetector(readout="x")

livekit_plugins_duplexjev-0.2.0/livekit/plugins/duplexjev/version.py

@@ -1 +0,0 @@
-__version__ = "0.2.0"