swarmauri_llm_hyperbolic 0.11.0.dev1__tar.gz → 0.11.0.dev4__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,7 +1,7 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: swarmauri_llm_hyperbolic
3
- Version: 0.11.0.dev1
4
- Summary: Provider-specific Swarmauri import package for Hyperbolic OpenAI-compatible chat, vision-language inference, streaming, and text-to-speech workflows.
3
+ Version: 0.11.0.dev4
4
+ Summary: Provider-specific Swarmauri import package for Hyperbolic OpenAI-compatible chat, vision-language inference, streaming, with retired audio imports retained for compatibility.
5
5
  License-Expression: Apache-2.0
6
6
  License-File: LICENSE
7
7
  Keywords: swarmauri,llm,hyperbolic,hyperbolic api,openai compatible,chat completions,vision language model,vlm,text to speech,tts,audio generation,streaming llm,async llm,qwen,llama,deepseek,large language model,generative ai,swarmauri llm,model,composable intelligence
@@ -28,7 +28,7 @@ Classifier: Programming Language :: Python :: 3.13
28
28
  Classifier: Programming Language :: Python :: 3.14
29
29
  Requires-Dist: swarmauri_base
30
30
  Requires-Dist: swarmauri_core
31
- Requires-Dist: swarmauri_standard
31
+ Requires-Dist: swarmauri_standard (>=0.11.0.dev12,<0.12)
32
32
  Description-Content-Type: text/markdown
33
33
 
34
34
  ![Swarmauri Logo](https://raw.githubusercontent.com/swarmauri/swarmauri-sdk/master/assets/swarmauri_sdk_brand.png)
@@ -49,70 +49,26 @@ Description-Content-Type: text/markdown
49
49
 
50
50
  # Swarmauri Hyperbolic LLM
51
51
 
52
- `swarmauri_llm_hyperbolic` provides provider-specific imports for Hyperbolic
53
- inference in Swarmauri applications. The package exports text chat,
54
- vision-language, and text-to-speech adapters while keeping implementation parity
55
- with `swarmauri_standard`.
52
+ `swarmauri_llm_hyperbolic` preserves Hyperbolic chat and vision imports and the
53
+ retired `HyperbolicAudioTTS` name. The old `.xyz/v1` API was decommissioned.
54
+ The current Hyperbolic inference service documents text and vision, with new
55
+ endpoints and credentials; it does not document a compatible audio replacement.
56
56
 
57
- Hyperbolic's inference documentation describes OpenAI-compatible chat
58
- completion APIs under `https://api.hyperbolic.xyz/v1/chat/completions`, model
59
- catalog discovery under `/v1/models`, vision-language models that accept images
60
- alongside text, and audio generation through `/v1/audio/generation`. This
61
- package maps those provider surfaces into Swarmauri components:
57
+ `HyperbolicAudioTTS` now warns and raises a local retirement error on construction
58
+ or execution. No audio request is sent. Select a standalone TTS adapter from the
59
+ [SDK migration guide](https://github.com/swarmauri/swarmauri-sdk/blob/master/docs/migrations/playht.md).
62
60
 
63
- - `HyperbolicModel` for OpenAI-compatible text chat completions.
64
- - `HyperbolicVisionModel` for vision-language chat completions with image input.
65
- - `HyperbolicAudioTTS` for text-to-speech audio generation.
66
-
67
- ## Why Use This Package?
68
-
69
- - Keep Hyperbolic-specific inference imports explicit in Swarmauri applications.
70
- - Use Swarmauri `Conversation` objects with Hyperbolic's OpenAI-compatible chat
71
- and vision-language endpoints.
72
- - Add text-to-speech generation without wiring raw base64 audio responses in
73
- application code.
74
- - Preserve a provider package boundary while relying on shared
75
- `swarmauri_standard` runtime implementations.
76
-
77
- ## FAQ
78
-
79
- ### What does `swarmauri_llm_hyperbolic` install?
80
-
81
- It installs provider package entry points for `HyperbolicModel`,
82
- `HyperbolicVisionModel`, and `HyperbolicAudioTTS`.
83
-
84
- ### Which Hyperbolic endpoints does it use?
85
-
86
- `HyperbolicModel` and `HyperbolicVisionModel` use Hyperbolic's
87
- OpenAI-compatible `chat/completions` endpoint. `HyperbolicAudioTTS` posts to
88
- `https://api.hyperbolic.xyz/v1/audio/generation` and writes the returned base64
89
- audio payload to a local file.
90
-
91
- ### Does it support streaming?
92
-
93
- `HyperbolicModel` supports synchronous and asynchronous streaming for chat
94
- completion responses. `HyperbolicAudioTTS` does not support streaming and raises
95
- `NotImplementedError` for `stream` and `astream`.
96
-
97
- ### Does it support vision-language prompts?
98
-
99
- Yes. `HyperbolicVisionModel` accepts Swarmauri message content lists with text
100
- and image input. Local image file paths can be converted to base64 data URLs by
101
- the runtime helper.
102
-
103
- ### Is the TTS adapter a general LLM?
104
-
105
- No. `HyperbolicAudioTTS` is a text-to-speech adapter kept in this provider
106
- package for compatibility. The runtime deprecation warning points new projects
107
- toward newer Swarmauri TTS imports when available.
61
+ The chat and vision implementations have not been migrated or live-verified in
62
+ this TTS update. Their presence here does not establish current availability.
63
+ See [Hyperbolic's migration notice](https://www.hyperbolic.ai/docs/inference/overview#moving-from-the-previous-inference-api).
108
64
 
109
65
  ## Features
110
66
 
111
67
  - `HyperbolicModel` for sync, async, streaming, and batch chat completion
112
68
  workflows.
113
69
  - `HyperbolicVisionModel` for image-plus-text prompts against Hyperbolic VLMs.
114
- - `HyperbolicAudioTTS` for synchronous and asynchronous text-to-speech
115
- generation.
70
+ - `HyperbolicAudioTTS` as an import-compatible retired type; it does not call
71
+ the decommissioned audio API.
116
72
  - Model discovery through Hyperbolic's `/models` endpoint.
117
73
  - Generation controls including `temperature`, `max_tokens`, `top_p`, `top_k`,
118
74
  and `stop`.
@@ -189,21 +145,12 @@ result = model.predict(conversation=conversation)
189
145
  print(result.get_last().content)
190
146
  ```
191
147
 
192
- ### Text To Speech
148
+ ### Text To Speech (retired)
193
149
 
194
- ```python
195
- from swarmauri_llm_hyperbolic import HyperbolicAudioTTS
196
-
197
- tts = HyperbolicAudioTTS(
198
- api_key="HYPERBOLIC_API_KEY",
199
- language="EN",
200
- speaker="EN-US",
201
- speed=1.0,
202
- )
203
-
204
- audio_path = tts.predict("Swarmauri turns model providers into components.")
205
- print(audio_path)
206
- ```
150
+ Do not construct HyperbolicAudioTTS or HyperbolicTTS: the legacy API was
151
+ decommissioned. Imports remain, but use emits a warning and raises a local
152
+ migration error. See [the SDK guide](../../../docs/migrations/playht.md) and
153
+ [Hyperbolic's migration notice](https://www.hyperbolic.ai/docs/inference/overview#moving-from-the-previous-inference-api).
207
154
 
208
155
  ## Related Packages
209
156
 
@@ -212,7 +159,7 @@ print(audio_path)
212
159
  - [swarmauri_llm_mistral](https://pypi.org/project/swarmauri_llm_mistral/)
213
160
  - [swarmauri_llm_openai](https://pypi.org/project/swarmauri_llm_openai/)
214
161
  - [swarmauri_llm_perplexity](https://pypi.org/project/swarmauri_llm_perplexity/)
215
- - [swarmauri_llm_playht](https://pypi.org/project/swarmauri_llm_playht/)
162
+ - [OpenAI TTS (Play.ht replacement)](../../standards/swarmauri_tts_openai/README.md)
216
163
 
217
164
  ## Foundational Swarmauri Packages
218
165
 
@@ -245,3 +192,7 @@ Apache-2.0
245
192
 
246
193
 
247
194
 
195
+ The audio retirement does not redirect API keys or rewrite the other LLM/VLM
196
+ components. Those legacy integrations need their own provider migration and
197
+ verification; this update does not claim their current hosted availability.
198
+
@@ -16,70 +16,26 @@
16
16
 
17
17
  # Swarmauri Hyperbolic LLM
18
18
 
19
- `swarmauri_llm_hyperbolic` provides provider-specific imports for Hyperbolic
20
- inference in Swarmauri applications. The package exports text chat,
21
- vision-language, and text-to-speech adapters while keeping implementation parity
22
- with `swarmauri_standard`.
19
+ `swarmauri_llm_hyperbolic` preserves Hyperbolic chat and vision imports and the
20
+ retired `HyperbolicAudioTTS` name. The old `.xyz/v1` API was decommissioned.
21
+ The current Hyperbolic inference service documents text and vision, with new
22
+ endpoints and credentials; it does not document a compatible audio replacement.
23
23
 
24
- Hyperbolic's inference documentation describes OpenAI-compatible chat
25
- completion APIs under `https://api.hyperbolic.xyz/v1/chat/completions`, model
26
- catalog discovery under `/v1/models`, vision-language models that accept images
27
- alongside text, and audio generation through `/v1/audio/generation`. This
28
- package maps those provider surfaces into Swarmauri components:
24
+ `HyperbolicAudioTTS` now warns and raises a local retirement error on construction
25
+ or execution. No audio request is sent. Select a standalone TTS adapter from the
26
+ [SDK migration guide](https://github.com/swarmauri/swarmauri-sdk/blob/master/docs/migrations/playht.md).
29
27
 
30
- - `HyperbolicModel` for OpenAI-compatible text chat completions.
31
- - `HyperbolicVisionModel` for vision-language chat completions with image input.
32
- - `HyperbolicAudioTTS` for text-to-speech audio generation.
33
-
34
- ## Why Use This Package?
35
-
36
- - Keep Hyperbolic-specific inference imports explicit in Swarmauri applications.
37
- - Use Swarmauri `Conversation` objects with Hyperbolic's OpenAI-compatible chat
38
- and vision-language endpoints.
39
- - Add text-to-speech generation without wiring raw base64 audio responses in
40
- application code.
41
- - Preserve a provider package boundary while relying on shared
42
- `swarmauri_standard` runtime implementations.
43
-
44
- ## FAQ
45
-
46
- ### What does `swarmauri_llm_hyperbolic` install?
47
-
48
- It installs provider package entry points for `HyperbolicModel`,
49
- `HyperbolicVisionModel`, and `HyperbolicAudioTTS`.
50
-
51
- ### Which Hyperbolic endpoints does it use?
52
-
53
- `HyperbolicModel` and `HyperbolicVisionModel` use Hyperbolic's
54
- OpenAI-compatible `chat/completions` endpoint. `HyperbolicAudioTTS` posts to
55
- `https://api.hyperbolic.xyz/v1/audio/generation` and writes the returned base64
56
- audio payload to a local file.
57
-
58
- ### Does it support streaming?
59
-
60
- `HyperbolicModel` supports synchronous and asynchronous streaming for chat
61
- completion responses. `HyperbolicAudioTTS` does not support streaming and raises
62
- `NotImplementedError` for `stream` and `astream`.
63
-
64
- ### Does it support vision-language prompts?
65
-
66
- Yes. `HyperbolicVisionModel` accepts Swarmauri message content lists with text
67
- and image input. Local image file paths can be converted to base64 data URLs by
68
- the runtime helper.
69
-
70
- ### Is the TTS adapter a general LLM?
71
-
72
- No. `HyperbolicAudioTTS` is a text-to-speech adapter kept in this provider
73
- package for compatibility. The runtime deprecation warning points new projects
74
- toward newer Swarmauri TTS imports when available.
28
+ The chat and vision implementations have not been migrated or live-verified in
29
+ this TTS update. Their presence here does not establish current availability.
30
+ See [Hyperbolic's migration notice](https://www.hyperbolic.ai/docs/inference/overview#moving-from-the-previous-inference-api).
75
31
 
76
32
  ## Features
77
33
 
78
34
  - `HyperbolicModel` for sync, async, streaming, and batch chat completion
79
35
  workflows.
80
36
  - `HyperbolicVisionModel` for image-plus-text prompts against Hyperbolic VLMs.
81
- - `HyperbolicAudioTTS` for synchronous and asynchronous text-to-speech
82
- generation.
37
+ - `HyperbolicAudioTTS` as an import-compatible retired type; it does not call
38
+ the decommissioned audio API.
83
39
  - Model discovery through Hyperbolic's `/models` endpoint.
84
40
  - Generation controls including `temperature`, `max_tokens`, `top_p`, `top_k`,
85
41
  and `stop`.
@@ -156,21 +112,12 @@ result = model.predict(conversation=conversation)
156
112
  print(result.get_last().content)
157
113
  ```
158
114
 
159
- ### Text To Speech
115
+ ### Text To Speech (retired)
160
116
 
161
- ```python
162
- from swarmauri_llm_hyperbolic import HyperbolicAudioTTS
163
-
164
- tts = HyperbolicAudioTTS(
165
- api_key="HYPERBOLIC_API_KEY",
166
- language="EN",
167
- speaker="EN-US",
168
- speed=1.0,
169
- )
170
-
171
- audio_path = tts.predict("Swarmauri turns model providers into components.")
172
- print(audio_path)
173
- ```
117
+ Do not construct HyperbolicAudioTTS or HyperbolicTTS: the legacy API was
118
+ decommissioned. Imports remain, but use emits a warning and raises a local
119
+ migration error. See [the SDK guide](../../../docs/migrations/playht.md) and
120
+ [Hyperbolic's migration notice](https://www.hyperbolic.ai/docs/inference/overview#moving-from-the-previous-inference-api).
174
121
 
175
122
  ## Related Packages
176
123
 
@@ -179,7 +126,7 @@ print(audio_path)
179
126
  - [swarmauri_llm_mistral](https://pypi.org/project/swarmauri_llm_mistral/)
180
127
  - [swarmauri_llm_openai](https://pypi.org/project/swarmauri_llm_openai/)
181
128
  - [swarmauri_llm_perplexity](https://pypi.org/project/swarmauri_llm_perplexity/)
182
- - [swarmauri_llm_playht](https://pypi.org/project/swarmauri_llm_playht/)
129
+ - [OpenAI TTS (Play.ht replacement)](../../standards/swarmauri_tts_openai/README.md)
183
130
 
184
131
  ## Foundational Swarmauri Packages
185
132
 
@@ -211,3 +158,7 @@ print(audio_path)
211
158
  Apache-2.0
212
159
 
213
160
 
161
+
162
+ The audio retirement does not redirect API keys or rewrite the other LLM/VLM
163
+ components. Those legacy integrations need their own provider migration and
164
+ verification; this update does not claim their current hosted availability.
@@ -1,7 +1,7 @@
1
1
  [project]
2
2
  name = "swarmauri_llm_hyperbolic"
3
- version = "0.11.0.dev1"
4
- description = "Provider-specific Swarmauri import package for Hyperbolic OpenAI-compatible chat, vision-language inference, streaming, and text-to-speech workflows."
3
+ version = "0.11.0.dev4"
4
+ description = "Provider-specific Swarmauri import package for Hyperbolic OpenAI-compatible chat, vision-language inference, streaming, with retired audio imports retained for compatibility."
5
5
  license = "Apache-2.0"
6
6
  readme = "README.md"
7
7
  repository = "http://github.com/swarmauri/swarmauri-sdk"
@@ -30,7 +30,7 @@ authors = [{ name = "Jacob Stewart", email = "jacob@swarmauri.com" }]
30
30
  dependencies = [
31
31
  "swarmauri_core",
32
32
  "swarmauri_base",
33
- "swarmauri_standard",
33
+ "swarmauri_standard>=0.11.0.dev12,<0.12",
34
34
  ]
35
35
  keywords = [
36
36
  "swarmauri",
@@ -0,0 +1,442 @@
1
+ import asyncio
2
+ import json
3
+ from typing import (
4
+ Any,
5
+ AsyncGenerator,
6
+ Dict,
7
+ Generator,
8
+ List,
9
+ Literal,
10
+ Optional,
11
+ Type,
12
+ )
13
+
14
+ import httpx
15
+ from pydantic import PrivateAttr, SecretStr
16
+ from swarmauri_base.ComponentBase import ComponentBase
17
+ from swarmauri_base.messages.MessageBase import MessageBase
18
+ from swarmauri_base.vlms.VLMBase import VLMBase
19
+
20
+ from swarmauri_standard.conversations.Conversation import Conversation
21
+ from swarmauri_standard.messages.AgentMessage import AgentMessage, UsageData
22
+ from swarmauri_standard.utils.file_path_to_base64 import file_path_to_base64
23
+ from swarmauri_standard.utils.retry_decorator import retry_on_status_codes
24
+
25
+
26
+ @ComponentBase.register_type(VLMBase, "HyperbolicVLM")
27
+ class HyperbolicVLM(VLMBase):
28
+ """
29
+ HyperbolicVLM class for interacting with the Hyperbolic vision language
30
+ models API. This class
31
+ provides synchronous and asynchronous methods to send conversation data to
32
+ the
33
+ model, receive predictions, and stream responses.
34
+
35
+ Attributes:
36
+ api_key (str): API key for authenticating requests to the Hyperbolic
37
+ API.
38
+ allowed_models (List[str]): List of allowed model names that can be
39
+ used.
40
+ name (str): The default model name to use for predictions.
41
+ type (Literal["HyperbolicVCM"]): The type identifier for this class.
42
+
43
+ Link to Allowed Models: https://app.hyperbolic.xyz/models
44
+ Link to API KEYS: https://app.hyperbolic.xyz/settings
45
+ """
46
+
47
+ api_key: SecretStr
48
+ allowed_models: List[str] = [
49
+ "Qwen/Qwen2.5-VL-72B-Instruct",
50
+ "Qwen/Qwen2.5-VL-7B-Instruct",
51
+ "mistralai/Pixtral-12B-2409",
52
+ ]
53
+ name: str = "Qwen/Qwen2.5-VL-72B-Instruct"
54
+ type: Literal["HyperbolicVisionModel"] = "HyperbolicVisionModel"
55
+ timeout: float = 600.0
56
+ _headers: Dict[str, str] = PrivateAttr(default=None)
57
+ _client: httpx.Client = PrivateAttr(default=None)
58
+ _BASE_URL: str = PrivateAttr(default="https://api.hyperbolic.xyz/v1/")
59
+
60
+ def __init__(self, **data: dict[str, Any]) -> None:
61
+ """
62
+ Initialize the HyperbolicVisionModel class with the provided data.
63
+
64
+ Args:
65
+ **data: Arbitrary keyword arguments containing initialization data.
66
+ """
67
+ super().__init__(**data)
68
+ self._headers = {
69
+ "Content-Type": "application/json",
70
+ "Authorization": f"Bearer {self.api_key.get_secret_value()}",
71
+ }
72
+ self._client = httpx.Client(
73
+ headers=self._headers,
74
+ base_url=self._BASE_URL,
75
+ timeout=self.timeout,
76
+ )
77
+
78
+ def _format_messages(
79
+ self,
80
+ messages: List[Type[MessageBase]],
81
+ ) -> List[Dict[str, Any]]:
82
+ """
83
+ Formats conversation messages into the structure expected by the API.
84
+
85
+ Args:
86
+ messages (List[MessageBase]): List of message objects from the
87
+ conversation history.
88
+
89
+ Returns:
90
+ List[Dict[str, Any]]: List of formatted message dictionaries.
91
+ """
92
+ formatted_messages = []
93
+ for message in messages:
94
+ formatted_message = message.model_dump(
95
+ include=["content", "role", "name"], exclude_none=True
96
+ )
97
+
98
+ if isinstance(formatted_message["content"], list):
99
+ formatted_content = []
100
+ for item in formatted_message["content"]:
101
+ if item["type"] == "image_url" and "file_path" in item:
102
+ # Convert file path to base64
103
+ base64_img = file_path_to_base64(item["file_path"])
104
+ formatted_content.append(
105
+ {
106
+ "type": "image_url",
107
+ "image_url": {
108
+ "url": (
109
+ f"data:image/jpeg;base64,{base64_img}"
110
+ )
111
+ },
112
+ }
113
+ )
114
+ else:
115
+ formatted_content.append(item)
116
+ formatted_message["content"] = formatted_content
117
+
118
+ formatted_messages.append(formatted_message)
119
+ return formatted_messages
120
+
121
+ def _prepare_usage_data(self, usage_data) -> UsageData:
122
+ """
123
+ Prepares and validates usage data received from the API response.
124
+
125
+ Args:
126
+ usage_data (dict): Raw usage data from the API response.
127
+
128
+ Returns:
129
+ UsageData: Validated usage data instance.
130
+ """
131
+ return UsageData.model_validate(usage_data)
132
+
133
+ @retry_on_status_codes((429, 529), max_retries=1)
134
+ def get_allowed_models(self) -> List[str]:
135
+ """
136
+ Get a list of allowed models for the Hyperbolic API.
137
+
138
+ Returns:
139
+ List[str]: List of allowed model names.
140
+ """
141
+ response = self._client.get("models")
142
+ response.raise_for_status()
143
+ response_data = response.json()
144
+
145
+ chat_models = [
146
+ model["id"]
147
+ for model in response_data["data"]
148
+ if model["supports_image_input"]
149
+ ]
150
+
151
+ return chat_models
152
+
153
+ @retry_on_status_codes((429, 529), max_retries=1)
154
+ def predict_vision(
155
+ self,
156
+ conversation: Conversation,
157
+ temperature: float = 0.7,
158
+ max_tokens: int = 2048,
159
+ top_p: float = 0.9,
160
+ stop: Optional[List[str]] = None,
161
+ ) -> Conversation:
162
+ """
163
+ Generates a response from the model based on the given conversation.
164
+
165
+ Args:
166
+ conversation (Conversation): Conversation object with message
167
+ history.
168
+ temperature (float): Sampling temperature for response diversity.
169
+ max_tokens (int): Maximum tokens for the model's response.
170
+ top_p (float): Cumulative probability for nucleus sampling.
171
+ stop (Optional[List[str]]): List of stop sequences for response
172
+ termination.
173
+
174
+ Returns:
175
+ Conversation: Updated conversation with the model's response.
176
+ """
177
+ formatted_messages = self._format_messages(conversation.history)
178
+ payload = {
179
+ "model": self.name,
180
+ "messages": formatted_messages,
181
+ "temperature": temperature,
182
+ "max_tokens": max_tokens,
183
+ "top_p": top_p,
184
+ "stop": stop or [],
185
+ }
186
+
187
+ response = self._client.post("chat/completions", json=payload)
188
+ response.raise_for_status()
189
+
190
+ response_data = response.json()
191
+
192
+ message_content = response_data["choices"][0]["message"]["content"]
193
+ usage_data = response_data.get("usage", {})
194
+
195
+ usage = self._prepare_usage_data(usage_data)
196
+ conversation.add_message(
197
+ AgentMessage(content=message_content, usage=usage)
198
+ )
199
+ return conversation
200
+
201
+ @retry_on_status_codes((429, 529), max_retries=1)
202
+ async def apredict_vision(
203
+ self,
204
+ conversation: Conversation,
205
+ temperature: float = 0.7,
206
+ max_tokens: int = 2048,
207
+ top_p: float = 0.9,
208
+ stop: Optional[List[str]] = None,
209
+ ) -> Conversation:
210
+ """
211
+ Async method to generate a response from the model based on the given
212
+ conversation.
213
+
214
+ Args:
215
+ conversation (Conversation): Conversation object with message
216
+ history.
217
+ temperature (float): Sampling temperature for response diversity.
218
+ max_tokens (int): Maximum tokens for the model's response.
219
+ top_p (float): Cumulative probability for nucleus sampling.
220
+ stop (Optional[List[str]]): List of stop sequences for response
221
+ termination.
222
+
223
+ Returns:
224
+ Conversation: Updated conversation with the model's response.
225
+ """
226
+ formatted_messages = self._format_messages(conversation.history)
227
+ payload = {
228
+ "model": self.name,
229
+ "messages": formatted_messages,
230
+ "temperature": temperature,
231
+ "max_tokens": max_tokens,
232
+ "top_p": top_p,
233
+ "stop": stop or [],
234
+ }
235
+
236
+ async with httpx.AsyncClient() as async_client:
237
+ response = await async_client.post(
238
+ f"{self._BASE_URL}chat/completions",
239
+ json=payload,
240
+ headers=self._headers,
241
+ )
242
+ response.raise_for_status()
243
+
244
+ response_data = response.json()
245
+
246
+ message_content = response_data["choices"][0]["message"]["content"]
247
+ usage_data = response_data.get("usage", {})
248
+
249
+ usage = self._prepare_usage_data(usage_data)
250
+ conversation.add_message(
251
+ AgentMessage(content=message_content, usage=usage)
252
+ )
253
+ return conversation
254
+
255
+ @retry_on_status_codes((429, 529), max_retries=1)
256
+ def stream(
257
+ self,
258
+ conversation: Conversation,
259
+ temperature: float = 0.7,
260
+ max_tokens: int = 2048,
261
+ top_p: float = 0.9,
262
+ stop: Optional[List[str]] = None,
263
+ ) -> Generator[str, None, None]:
264
+ """
265
+ Streams response text from the model in real-time.
266
+
267
+ Args:
268
+ conversation (Conversation): Conversation object with message
269
+ history.
270
+ temperature (float): Sampling temperature for response diversity.
271
+ max_tokens (int): Maximum tokens for the model's response.
272
+ top_p (float): Cumulative probability for nucleus sampling.
273
+ stop (Optional[List[str]]): List of stop sequences for response
274
+ termination.
275
+
276
+ Yields:
277
+ str: Partial response content from the model.
278
+ """
279
+ formatted_messages = self._format_messages(conversation.history)
280
+ payload = {
281
+ "model": self.name,
282
+ "messages": formatted_messages,
283
+ "temperature": temperature,
284
+ "max_tokens": max_tokens,
285
+ "top_p": top_p,
286
+ "stream": True,
287
+ "stop": stop or [],
288
+ }
289
+
290
+ response = self._client.post(self._BASE_URL, json=payload)
291
+ response.raise_for_status()
292
+
293
+ message_content = ""
294
+ for line in response.iter_lines():
295
+ json_str = line.replace("data: ", "")
296
+ try:
297
+ if json_str:
298
+ chunk = json.loads(json_str)
299
+ if chunk["choices"][0]["delta"]:
300
+ delta = chunk["choices"][0]["delta"]["content"]
301
+ message_content += delta
302
+ yield delta
303
+ except json.JSONDecodeError:
304
+ pass
305
+
306
+ conversation.add_message(AgentMessage(content=message_content))
307
+
308
+ @retry_on_status_codes((429, 529), max_retries=1)
309
+ async def astream(
310
+ self,
311
+ conversation: Conversation,
312
+ temperature: float = 0.7,
313
+ max_tokens: int = 2048,
314
+ top_p: float = 0.9,
315
+ stop: Optional[List[str]] = None,
316
+ ) -> AsyncGenerator[str, None]:
317
+ """
318
+ Async generator that streams response text from the model in real-time.
319
+
320
+ Args:
321
+ conversation (Conversation): Conversation object with message
322
+ history.
323
+ temperature (float): Sampling temperature for response diversity.
324
+ max_tokens (int): Maximum tokens for the model's response.
325
+ top_p (float): Cumulative probability for nucleus sampling.
326
+ stop (Optional[List[str]]): List of stop sequences for response
327
+ termination.
328
+
329
+ Yields:
330
+ str: Partial response content from the model.
331
+ """
332
+ formatted_messages = self._format_messages(conversation.history)
333
+ payload = {
334
+ "model": self.name,
335
+ "messages": formatted_messages,
336
+ "temperature": temperature,
337
+ "max_tokens": max_tokens,
338
+ "top_p": top_p,
339
+ "stream": True,
340
+ "stop": stop or [],
341
+ }
342
+
343
+ async with httpx.AsyncClient as async_client:
344
+ response = await async_client.post(
345
+ f"{self._BASE_URL}chat/completions",
346
+ json=payload,
347
+ headers=self._headers,
348
+ )
349
+ response.raise_for_status()
350
+
351
+ message_content = ""
352
+ async for line in response.aiter_lines():
353
+ json_str = line.replace("data: ", "")
354
+ try:
355
+ if json_str:
356
+ chunk = json.loads(json_str)
357
+ if chunk["choices"][0]["delta"]:
358
+ delta = chunk["choices"][0]["delta"]["content"]
359
+ message_content += delta
360
+ yield delta
361
+ except json.JSONDecodeError:
362
+ pass
363
+
364
+ conversation.add_message(AgentMessage(content=message_content))
365
+
366
+ def batch(
367
+ self,
368
+ conversations: List[Conversation],
369
+ temperature: float = 0.7,
370
+ max_tokens: int = 2048,
371
+ top_p: float = 0.9,
372
+ stop: Optional[List[str]] = None,
373
+ ) -> List[Conversation]:
374
+ """
375
+ Processes a batch of conversations and generates responses for each
376
+ sequentially.
377
+
378
+ Args:
379
+ conversations (List[Conversation]): List of conversations to
380
+ process.
381
+ temperature (float): Sampling temperature for response diversity.
382
+ max_tokens (int): Maximum tokens for each response.
383
+ top_p (float): Cumulative probability for nucleus sampling.
384
+ stop (Optional[List[str]]): List of stop sequences for response
385
+ termination.
386
+
387
+ Returns:
388
+ List[Conversation]: List of updated conversations with model
389
+ responses.
390
+ """
391
+ results = []
392
+ for conversation in conversations:
393
+ result_conversation = self.predict_vision(
394
+ conversation,
395
+ temperature=temperature,
396
+ max_tokens=max_tokens,
397
+ top_p=top_p,
398
+ stop=stop,
399
+ )
400
+ results.append(result_conversation)
401
+ return results
402
+
403
+ async def abatch(
404
+ self,
405
+ conversations: List[Conversation],
406
+ temperature: float = 0.7,
407
+ max_tokens: int = 2048,
408
+ top_p: float = 0.9,
409
+ stop: Optional[List[str]] = None,
410
+ max_concurrent=5,
411
+ ) -> List[Conversation]:
412
+ """
413
+ Async method for processing a batch of conversations concurrently.
414
+
415
+ Args:
416
+ conversations (List[Conversation]): List of conversations to
417
+ process.
418
+ temperature (float): Sampling temperature for response diversity.
419
+ max_tokens (int): Maximum tokens for each response.
420
+ top_p (float): Cumulative probability for nucleus sampling.
421
+ stop (Optional[List[str]]): List of stop sequences for response
422
+ termination.
423
+ max_concurrent (int): Maximum number of concurrent requests.
424
+
425
+ Returns:
426
+ List[Conversation]: List of updated conversations with model
427
+ responses.
428
+ """
429
+ semaphore = asyncio.Semaphore(max_concurrent)
430
+
431
+ async def process_conversation(conv: Conversation) -> Conversation:
432
+ async with semaphore:
433
+ return await self.apredict_vision(
434
+ conv,
435
+ temperature=temperature,
436
+ max_tokens=max_tokens,
437
+ top_p=top_p,
438
+ stop=stop,
439
+ )
440
+
441
+ tasks = [process_conversation(conv) for conv in conversations]
442
+ return await asyncio.gather(*tasks)
@@ -1,8 +1,14 @@
1
1
  from .HyperbolicAudioTTS import HyperbolicAudioTTS
2
2
  from .HyperbolicModel import HyperbolicModel
3
3
  from .HyperbolicVisionModel import HyperbolicVisionModel
4
+ from .HyperbolicVLM import HyperbolicVLM
4
5
 
5
- __all__ = ["HyperbolicAudioTTS", "HyperbolicModel", "HyperbolicVisionModel"]
6
+ __all__ = [
7
+ "HyperbolicAudioTTS",
8
+ "HyperbolicModel",
9
+ "HyperbolicVisionModel",
10
+ "HyperbolicVLM",
11
+ ]
6
12
 
7
13
  try:
8
14
  from importlib.metadata import PackageNotFoundError, version