livekit-plugins-chatfx 0.1.0__tar.gz

livekit_plugins_chatfx-0.1.0/.gitignore

@@ -0,0 +1,4 @@
+__pycache__/
+*.egg-info/
+dist/
+.pytest_cache/

livekit_plugins_chatfx-0.1.0/PKG-INFO

@@ -0,0 +1,132 @@
+Metadata-Version: 2.5
+Name: livekit-plugins-chatfx
+Version: 0.1.0
+Summary: ChatFX Avatar for LiveKit Agents: a real-time talking face for your voice agent
+Project-URL: Homepage, https://chatfx.com/avatar/
+Project-URL: Documentation, https://chatfx.com/support/avatar-connect/
+Author-email: "ChatRep, Inc." <service@chatfx.com>
+License-Expression: Apache-2.0
+Keywords: agents,avatar,chatfx,livekit,realtime,voice
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Multimedia :: Video
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Requires-Dist: aiohttp>=3.9
+Requires-Dist: livekit-agents>=1.2
+Requires-Dist: pillow>=10.1
+Description-Content-Type: text/markdown
+
+# ChatFX Avatar for LiveKit Agents
+
+A real-time talking face for your LiveKit voice agent. The avatar joins your agent's LiveKit Cloud room as one more
+participant, takes your agent's voice and speaks it with its lips in time, shown as your agent. You keep your own
+listening, AI and voice; ChatFX adds the face, billed by the minute.
+
+## Install
+
+```bash
+pip install livekit-plugins-chatfx
+```
+
+Requires Python 3.10+ and `livekit-agents` 1.2 or newer.
+
+## Add your key
+
+Create a key under **API keys** at [avatar.chatfx.com](https://avatar.chatfx.com). Then add it to your voice agent's secrets
+at LiveKit, named exactly `CHATFX_AVATAR_API_KEY`. In your agent's folder:
+
+```bash
+lk agent update-secrets --secrets "CHATFX_AVATAR_API_KEY=cfx_..."
+```
+
+Running your agent on your own server instead? Add the key to that server's environment variables, with the same name.
+
+Your agent also needs `LIVEKIT_URL`, `LIVEKIT_API_KEY` and `LIVEKIT_API_SECRET`, which it already has. They never leave your
+server.
+
+## Use it
+
+In your agent's entrypoint, after you make the `AgentSession` and before `session.start()`:
+
+```python
+from livekit.plugins import chatfx
+
+avatar = chatfx.AvatarSession(avatar_id="chatfx-lois")  # copy the id from the Avatars page
+await avatar.start(session, room=ctx.room)
+
+await session.start(agent=MyAgent(), room=ctx.room)
+```
+
+That's all the code. When the job ends, the session ends too, so billing stops. To end it earlier, call
+`await avatar.aclose()`.
+
+## What it does for you
+
+- **Shows a waiting picture, with nothing to add to your page.** The moment a call starts, your agent shows the avatar's picture
+  as its own video, with "Video starting..." on it, and the live face takes over when it's ready. Any page that shows the
+  agent's video gets this.
+- **Greets with the face.** `start()` waits up to 10 seconds for the face, so your agent's first words come with it.
+- **Starts by voice, over the picture, if the face is late.** The voice moves to the face between replies, so a sentence is never split.
+- **Never depends on the face.** If the face can't come, or leaves mid-call, your agent carries on by voice.
+- **Keeps trying** while a GPU wakes or is busy, for up to 5 minutes, and stops at once on a refusal more tries won't fix.
+- **Stops forgotten calls**, so they don't use your minutes: after 2 minutes with nobody talking, 2 minutes after the visitor
+  leaves (long enough for a page reload), or at 1 hour.
+- **Can't go deaf.** If the face's "finished talking" message is late, the plugin ends the reply once the sound has run out,
+  so your agent hears the next question.
+
+## Options
+
+```python
+chatfx.AvatarSession(
+    avatar_id="chatfx-lois",
+    api_key=None,                           # defaults to CHATFX_AVATAR_API_KEY
+    api_url=None,                           # defaults to https://api.chatfx.com
+    avatar_participant_identity="chatfx-avatar",
+    avatar_participant_name="ChatFX",
+    face_first_seconds=10.0,                # how long start() waits for the face
+    start_timeout=300.0,                    # how long to keep trying while a GPU wakes
+    quiet_seconds=120.0,                    # end the call after this long with nobody talking (None: never)
+    visitor_gone_seconds=120.0,             # end the call this long after the last visitor leaves (None: never)
+    longest_call_seconds=3600.0,            # end the call at this length (None: never)
+    end_call_when_idle=True,                # False: only the avatar leaves, and your agent carries on by voice
+    waiting_picture=True,                   # show the avatar's picture until the live face is ready
+    waiting_text="Video starting...",        # the words on it, in your language (None: no words)
+)
+```
+
+`avatar.start()` also takes `livekit_url`, `livekit_api_key` and `livekit_api_secret` if you'd rather not use the settings.
+
+We recommend `"interruption": {"min_words": 1}` in your session's turn handling. The avatar's sound can't pause and carry on,
+so a false alarm would otherwise cut a reply off for good.
+
+## Optional: make the face appear sooner
+
+The plugin starts waking the GPU as soon as a call begins, which takes about 11 seconds. To start even earlier, call `warm()` from your website's server when a visitor
+is about to start a call, and again every 20 seconds while they get ready. It's free. Your website's server then needs the key
+in its environment variables too. Never put the key in a web page.
+
+```python
+await chatfx.warm()
+```
+
+Or without Python:
+
+```bash
+curl -X POST https://api.chatfx.com/v1/warm -H "Authorization: Bearer $CHATFX_AVATAR_API_KEY"
+```
+
+## If you build your own call screen
+
+Your agent's participant attribute `chatfx-face` says where the face is: `waking`, `voice`, `on` or `off`. In the agent,
+`avatar.face_status` says the same. You don't need either to get the waiting picture.
+
+## When there's no face
+
+A refusal doesn't stop the call. The log says why, and `avatar.failure` is a `chatfx.ChatFXException` with the API's `code`
+(for example `invalid_api_key`, `avatar_not_found`, `premium_avatar`, `out_of_minutes`) and a plain message. A missing key or
+missing LiveKit setting raises right away, so you find it while setting up.
+
+## Docs
+
+[Connecting your agent](https://chatfx.com/support/avatar-connect/) · [Building avatars](https://chatfx.com/support/avatar-builder/) · [Minutes and plans](https://chatfx.com/support/avatar-billing/)

livekit_plugins_chatfx-0.1.0/README.md

@@ -0,0 +1,113 @@
+# ChatFX Avatar for LiveKit Agents
+
+A real-time talking face for your LiveKit voice agent. The avatar joins your agent's LiveKit Cloud room as one more
+participant, takes your agent's voice and speaks it with its lips in time, shown as your agent. You keep your own
+listening, AI and voice; ChatFX adds the face, billed by the minute.
+
+## Install
+
+```bash
+pip install livekit-plugins-chatfx
+```
+
+Requires Python 3.10+ and `livekit-agents` 1.2 or newer.
+
+## Add your key
+
+Create a key under **API keys** at [avatar.chatfx.com](https://avatar.chatfx.com). Then add it to your voice agent's secrets
+at LiveKit, named exactly `CHATFX_AVATAR_API_KEY`. In your agent's folder:
+
+```bash
+lk agent update-secrets --secrets "CHATFX_AVATAR_API_KEY=cfx_..."
+```
+
+Running your agent on your own server instead? Add the key to that server's environment variables, with the same name.
+
+Your agent also needs `LIVEKIT_URL`, `LIVEKIT_API_KEY` and `LIVEKIT_API_SECRET`, which it already has. They never leave your
+server.
+
+## Use it
+
+In your agent's entrypoint, after you make the `AgentSession` and before `session.start()`:
+
+```python
+from livekit.plugins import chatfx
+
+avatar = chatfx.AvatarSession(avatar_id="chatfx-lois")  # copy the id from the Avatars page
+await avatar.start(session, room=ctx.room)
+
+await session.start(agent=MyAgent(), room=ctx.room)
+```
+
+That's all the code. When the job ends, the session ends too, so billing stops. To end it earlier, call
+`await avatar.aclose()`.
+
+## What it does for you
+
+- **Shows a waiting picture, with nothing to add to your page.** The moment a call starts, your agent shows the avatar's picture
+  as its own video, with "Video starting..." on it, and the live face takes over when it's ready. Any page that shows the
+  agent's video gets this.
+- **Greets with the face.** `start()` waits up to 10 seconds for the face, so your agent's first words come with it.
+- **Starts by voice, over the picture, if the face is late.** The voice moves to the face between replies, so a sentence is never split.
+- **Never depends on the face.** If the face can't come, or leaves mid-call, your agent carries on by voice.
+- **Keeps trying** while a GPU wakes or is busy, for up to 5 minutes, and stops at once on a refusal more tries won't fix.
+- **Stops forgotten calls**, so they don't use your minutes: after 2 minutes with nobody talking, 2 minutes after the visitor
+  leaves (long enough for a page reload), or at 1 hour.
+- **Can't go deaf.** If the face's "finished talking" message is late, the plugin ends the reply once the sound has run out,
+  so your agent hears the next question.
+
+## Options
+
+```python
+chatfx.AvatarSession(
+    avatar_id="chatfx-lois",
+    api_key=None,                           # defaults to CHATFX_AVATAR_API_KEY
+    api_url=None,                           # defaults to https://api.chatfx.com
+    avatar_participant_identity="chatfx-avatar",
+    avatar_participant_name="ChatFX",
+    face_first_seconds=10.0,                # how long start() waits for the face
+    start_timeout=300.0,                    # how long to keep trying while a GPU wakes
+    quiet_seconds=120.0,                    # end the call after this long with nobody talking (None: never)
+    visitor_gone_seconds=120.0,             # end the call this long after the last visitor leaves (None: never)
+    longest_call_seconds=3600.0,            # end the call at this length (None: never)
+    end_call_when_idle=True,                # False: only the avatar leaves, and your agent carries on by voice
+    waiting_picture=True,                   # show the avatar's picture until the live face is ready
+    waiting_text="Video starting...",        # the words on it, in your language (None: no words)
+)
+```
+
+`avatar.start()` also takes `livekit_url`, `livekit_api_key` and `livekit_api_secret` if you'd rather not use the settings.
+
+We recommend `"interruption": {"min_words": 1}` in your session's turn handling. The avatar's sound can't pause and carry on,
+so a false alarm would otherwise cut a reply off for good.
+
+## Optional: make the face appear sooner
+
+The plugin starts waking the GPU as soon as a call begins, which takes about 11 seconds. To start even earlier, call `warm()` from your website's server when a visitor
+is about to start a call, and again every 20 seconds while they get ready. It's free. Your website's server then needs the key
+in its environment variables too. Never put the key in a web page.
+
+```python
+await chatfx.warm()
+```
+
+Or without Python:
+
+```bash
+curl -X POST https://api.chatfx.com/v1/warm -H "Authorization: Bearer $CHATFX_AVATAR_API_KEY"
+```
+
+## If you build your own call screen
+
+Your agent's participant attribute `chatfx-face` says where the face is: `waking`, `voice`, `on` or `off`. In the agent,
+`avatar.face_status` says the same. You don't need either to get the waiting picture.
+
+## When there's no face
+
+A refusal doesn't stop the call. The log says why, and `avatar.failure` is a `chatfx.ChatFXException` with the API's `code`
+(for example `invalid_api_key`, `avatar_not_found`, `premium_avatar`, `out_of_minutes`) and a plain message. A missing key or
+missing LiveKit setting raises right away, so you find it while setting up.
+
+## Docs
+
+[Connecting your agent](https://chatfx.com/support/avatar-connect/) · [Building avatars](https://chatfx.com/support/avatar-builder/) · [Minutes and plans](https://chatfx.com/support/avatar-billing/)

livekit_plugins_chatfx-0.1.0/livekit/plugins/chatfx/__init__.py

@@ -0,0 +1,37 @@
+"""ChatFX Avatar for LiveKit Agents.
+
+A real-time talking face for your voice agent. The avatar joins your agent's LiveKit room as one more participant, takes
+your agent's voice and speaks it with its lips in time, shown as your agent.
+
+    from livekit.plugins import chatfx
+
+    avatar = chatfx.AvatarSession(avatar_id="chatfx-lois")  # your key in CHATFX_AVATAR_API_KEY
+    await avatar.start(session, room=ctx.room)
+
+Docs: https://chatfx.com/support/avatar-connect/
+"""
+
+from livekit.agents import Plugin
+
+from .avatar import AvatarSession, ChatFXException, warm
+from .log import logger
+from .version import __version__
+
+__all__ = ["AvatarSession", "ChatFXException", "warm", "__version__"]
+
+
+class ChatFXPlugin(Plugin):
+    def __init__(self) -> None:
+        super().__init__(__name__, __version__, __package__, logger)
+
+
+Plugin.register_plugin(ChatFXPlugin())
+
+# Cleanup docs of unexported modules
+_module = dir()
+NOT_IN_ALL = [m for m in _module if m not in __all__]
+
+__pdoc__ = {}
+
+for n in NOT_IN_ALL:
+    __pdoc__[n] = False