guidellm 0.3.0rc20250418__tar.gz → 0.3.0rc20250425__tar.gz

{guidellm-0.3.0rc20250418/src/guidellm.egg-info → guidellm-0.3.0rc20250425}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: guidellm
-Version: 0.3.0rc20250418
+Version: 0.3.0rc20250425
 Summary: Guidance platform for deploying and managing large language models.
 Author: Neuralmagic, Inc.
 License:                                  Apache License
@@ -259,7 +259,7 @@ Dynamic: license-file
 Scale Efficiently: Evaluate and Optimize Your LLM Deployments for Real-World Inference
 </h3>
 
-[![GitHub Release](https://img.shields.io/github/release/neuralmagic/guidellm.svg?label=Version)](https://github.com/neuralmagic/guidellm/releases) [![Documentation](https://img.shields.io/badge/Documentation-8A2BE2?logo=read-the-docs&logoColor=%23ffffff&color=%231BC070)](https://github.com/neuralmagic/guidellm/tree/main/docs) [![License](https://img.shields.io/github/license/neuralmagic/guidellm.svg)](https://github.com/neuralmagic/guidellm/blob/main/LICENSE) [![PyPI Release](https://img.shields.io/pypi/v/guidellm.svg?label=PyPI%20Release)](https://pypi.python.org/pypi/guidellm) [![Python Versions](https://img.shields.io/badge/Python-3.8--3.12-orange)](https://pypi.python.org/pypi/guidellm) [![Nightly Build](https://img.shields.io/github/actions/workflow/status/neuralmagic/guidellm/nightly.yml?branch=main&label=Nightly%20Build)](https://github.com/neuralmagic/guidellm/actions/workflows/nightly.yml)
+[![GitHub Release](https://img.shields.io/github/release/neuralmagic/guidellm.svg?label=Version)](https://github.com/neuralmagic/guidellm/releases) [![Documentation](https://img.shields.io/badge/Documentation-8A2BE2?logo=read-the-docs&logoColor=%23ffffff&color=%231BC070)](https://github.com/neuralmagic/guidellm/tree/main/docs) [![License](https://img.shields.io/github/license/neuralmagic/guidellm.svg)](https://github.com/neuralmagic/guidellm/blob/main/LICENSE) [![PyPI Release](https://img.shields.io/pypi/v/guidellm.svg?label=PyPI%20Release)](https://pypi.python.org/pypi/guidellm) [![Python Versions](https://img.shields.io/badge/Python-3.9--3.13-orange)](https://pypi.python.org/pypi/guidellm) [![Nightly Build](https://img.shields.io/github/actions/workflow/status/neuralmagic/guidellm/nightly.yml?branch=main&label=Nightly%20Build)](https://github.com/neuralmagic/guidellm/actions/workflows/nightly.yml)
 
 ## Overview
 
@@ -413,6 +413,7 @@ Our comprehensive documentation offers detailed guides and resources to help you
 
 - [**Installation Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/install.md) - This guide provides step-by-step instructions for installing GuideLLM, including prerequisites and setup tips.
 - [**Backends Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/backends.md) - A comprehensive overview of supported backends and how to set them up for use with GuideLLM.
+- [**Data/Datasets Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/datasets.md) - Information on supported datasets, including how to use them for benchmarking.
 - [**Metrics Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/metrics.md) - Detailed explanations of the metrics used in GuideLLM, including definitions and how to interpret them.
 - [**Outputs Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/outputs.md) - Information on the different output formats supported by GuideLLM and how to use them.
 - [**Architecture Overview**](https://github.com/neuralmagic/guidellm/tree/main/docs/architecture.md) - A detailed look at GuideLLM's design, components, and how they interact.

{guidellm-0.3.0rc20250418 → guidellm-0.3.0rc20250425}/README.md

@@ -9,7 +9,7 @@
 Scale Efficiently: Evaluate and Optimize Your LLM Deployments for Real-World Inference
 </h3>
 
-[![GitHub Release](https://img.shields.io/github/release/neuralmagic/guidellm.svg?label=Version)](https://github.com/neuralmagic/guidellm/releases) [![Documentation](https://img.shields.io/badge/Documentation-8A2BE2?logo=read-the-docs&logoColor=%23ffffff&color=%231BC070)](https://github.com/neuralmagic/guidellm/tree/main/docs) [![License](https://img.shields.io/github/license/neuralmagic/guidellm.svg)](https://github.com/neuralmagic/guidellm/blob/main/LICENSE) [![PyPI Release](https://img.shields.io/pypi/v/guidellm.svg?label=PyPI%20Release)](https://pypi.python.org/pypi/guidellm) [![Python Versions](https://img.shields.io/badge/Python-3.8--3.12-orange)](https://pypi.python.org/pypi/guidellm) [![Nightly Build](https://img.shields.io/github/actions/workflow/status/neuralmagic/guidellm/nightly.yml?branch=main&label=Nightly%20Build)](https://github.com/neuralmagic/guidellm/actions/workflows/nightly.yml)
+[![GitHub Release](https://img.shields.io/github/release/neuralmagic/guidellm.svg?label=Version)](https://github.com/neuralmagic/guidellm/releases) [![Documentation](https://img.shields.io/badge/Documentation-8A2BE2?logo=read-the-docs&logoColor=%23ffffff&color=%231BC070)](https://github.com/neuralmagic/guidellm/tree/main/docs) [![License](https://img.shields.io/github/license/neuralmagic/guidellm.svg)](https://github.com/neuralmagic/guidellm/blob/main/LICENSE) [![PyPI Release](https://img.shields.io/pypi/v/guidellm.svg?label=PyPI%20Release)](https://pypi.python.org/pypi/guidellm) [![Python Versions](https://img.shields.io/badge/Python-3.9--3.13-orange)](https://pypi.python.org/pypi/guidellm) [![Nightly Build](https://img.shields.io/github/actions/workflow/status/neuralmagic/guidellm/nightly.yml?branch=main&label=Nightly%20Build)](https://github.com/neuralmagic/guidellm/actions/workflows/nightly.yml)
 
 ## Overview
 
@@ -163,6 +163,7 @@ Our comprehensive documentation offers detailed guides and resources to help you
 
 - [**Installation Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/install.md) - This guide provides step-by-step instructions for installing GuideLLM, including prerequisites and setup tips.
 - [**Backends Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/backends.md) - A comprehensive overview of supported backends and how to set them up for use with GuideLLM.
+- [**Data/Datasets Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/datasets.md) - Information on supported datasets, including how to use them for benchmarking.
 - [**Metrics Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/metrics.md) - Detailed explanations of the metrics used in GuideLLM, including definitions and how to interpret them.
 - [**Outputs Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/outputs.md) - Information on the different output formats supported by GuideLLM and how to use them.
 - [**Architecture Overview**](https://github.com/neuralmagic/guidellm/tree/main/docs/architecture.md) - A detailed look at GuideLLM's design, components, and how they interact.

{guidellm-0.3.0rc20250418 → guidellm-0.3.0rc20250425}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "guidellm"
-version = "0.3.0.rc20250418"
+version = "0.3.0.rc20250425"
 description = "Guidance platform for deploying and managing large language models."
 requires-python = ">=3.9.0,<4.0"
 dependencies = [ "click", "datasets", "ftfy>=6.0.0", "httpx[http2]<1.0.0", "loguru", "numpy", "pillow", "protobuf", "pydantic>=2.0.0", "pydantic-settings>=2.0.0", "pyyaml>=6.0.0", "requests", "rich", "transformers",]

{guidellm-0.3.0rc20250418 → guidellm-0.3.0rc20250425}/src/guidellm/backend/openai.py

@@ -145,7 +145,7 @@ class OpenAIHTTPBackend(Backend):
         elif self.model not in models:
             raise ValueError(
                 f"Model {self.model} not found in available models:"
-                "{models} for target: {self.target}"
+                f"{models} for target: {self.target}"
             )
 
     async def prepare_multiprocessing(self):
@@ -202,6 +202,13 @@ class OpenAIHTTPBackend(Backend):
             and a ResponseSummary for the final response.
         """
         logger.debug("{} invocation with args: {}", self.__class__.__name__, locals())
+
+        if isinstance(prompt, list):
+            raise ValueError(
+                "List prompts (batching) is currently not supported for "
+                f"text_completions OpenAI pathways. Received: {prompt}"
+            )
+
         headers = self._headers()
         payload = self._completions_payload(
             orig_kwargs=kwargs,

{guidellm-0.3.0rc20250418 → guidellm-0.3.0rc20250425}/src/guidellm/config.py

@@ -134,6 +134,9 @@ class Settings(BaseSettings):
         Literal["request", "response", "local"]
     ] = "response"
     preferred_backend: Literal["openai"] = "openai"
+    preferred_route: Literal["text_completions", "chat_completions"] = (
+        "text_completions"
+    )
     openai: OpenAISettings = OpenAISettings()
 
     # Output settings

{guidellm-0.3.0rc20250418 → guidellm-0.3.0rc20250425}/src/guidellm/dataset/file.py

@@ -71,19 +71,21 @@ class FileDatasetCreator(DatasetCreator):
 
             dataset = Dataset.from_dict({"text": items}, **(data_args or {}))
         elif path.suffix.lower() == ".csv":
-            dataset = load_dataset("csv", data_files=path, **(data_args or {}))
+            dataset = load_dataset("csv", data_files=str(path), **(data_args or {}))
         elif path.suffix.lower() in {".json", ".jsonl"}:
-            dataset = load_dataset("json", data_files=path, **(data_args or {}))
+            dataset = load_dataset("json", data_files=str(path), **(data_args or {}))
         elif path.suffix.lower() == ".parquet":
-            dataset = load_dataset("parquet", data_files=path, **(data_args or {}))
+            dataset = load_dataset("parquet", data_files=str(path), **(data_args or {}))
         elif path.suffix.lower() == ".arrow":
-            dataset = load_dataset("arrow", data_files=path, **(data_args or {}))
+            dataset = load_dataset("arrow", data_files=str(path), **(data_args or {}))
         elif path.suffix.lower() == ".hdf5":
-            dataset = Dataset.from_pandas(pd.read_hdf(path), **(data_args or {}))
+            dataset = Dataset.from_pandas(pd.read_hdf(str(path)), **(data_args or {}))
         elif path.suffix.lower() == ".db":
-            dataset = Dataset.from_sql(con=path, **(data_args or {}))
+            dataset = Dataset.from_sql(con=str(path), **(data_args or {}))
         elif path.suffix.lower() == ".tar":
-            dataset = load_dataset("webdataset", data_files=path, **(data_args or {}))
+            dataset = load_dataset(
+                "webdataset", data_files=str(path), **(data_args or {})
+            )
         else:
             raise ValueError(f"Unsupported file type: {path.suffix} given for {path}. ")
 

{guidellm-0.3.0rc20250418 → guidellm-0.3.0rc20250425}/src/guidellm/request/loader.py

@@ -11,6 +11,7 @@ from typing import (
 from datasets import Dataset, DatasetDict, IterableDataset, IterableDatasetDict
 from transformers import PreTrainedTokenizerBase  # type: ignore[import]
 
+from guidellm.config import settings
 from guidellm.dataset import ColumnInputTypes, load_dataset
 from guidellm.objects import StandardBaseModel
 from guidellm.request.request import GenerationRequest
@@ -61,6 +62,8 @@ class GenerativeRequestLoader(RequestLoader):
         "content",
         "conversation",
         "conversations",
+        "turn",
+        "turns",
         "text",
     ]
 
@@ -270,7 +273,7 @@ class GenerativeRequestLoader(RequestLoader):
         )
 
         return GenerationRequest(
-            request_type="text_completions",
+            request_type=settings.preferred_route,
             content=item[self.column_mappings["prompt_column"]],
             stats=(
                 {"prompt_tokens": prompt_tokens} if prompt_tokens is not None else {}

{guidellm-0.3.0rc20250418 → guidellm-0.3.0rc20250425/src/guidellm.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: guidellm
-Version: 0.3.0rc20250418
+Version: 0.3.0rc20250425
 Summary: Guidance platform for deploying and managing large language models.
 Author: Neuralmagic, Inc.
 License:                                  Apache License
@@ -259,7 +259,7 @@ Dynamic: license-file
 Scale Efficiently: Evaluate and Optimize Your LLM Deployments for Real-World Inference
 </h3>
 
-[![GitHub Release](https://img.shields.io/github/release/neuralmagic/guidellm.svg?label=Version)](https://github.com/neuralmagic/guidellm/releases) [![Documentation](https://img.shields.io/badge/Documentation-8A2BE2?logo=read-the-docs&logoColor=%23ffffff&color=%231BC070)](https://github.com/neuralmagic/guidellm/tree/main/docs) [![License](https://img.shields.io/github/license/neuralmagic/guidellm.svg)](https://github.com/neuralmagic/guidellm/blob/main/LICENSE) [![PyPI Release](https://img.shields.io/pypi/v/guidellm.svg?label=PyPI%20Release)](https://pypi.python.org/pypi/guidellm) [![Python Versions](https://img.shields.io/badge/Python-3.8--3.12-orange)](https://pypi.python.org/pypi/guidellm) [![Nightly Build](https://img.shields.io/github/actions/workflow/status/neuralmagic/guidellm/nightly.yml?branch=main&label=Nightly%20Build)](https://github.com/neuralmagic/guidellm/actions/workflows/nightly.yml)
+[![GitHub Release](https://img.shields.io/github/release/neuralmagic/guidellm.svg?label=Version)](https://github.com/neuralmagic/guidellm/releases) [![Documentation](https://img.shields.io/badge/Documentation-8A2BE2?logo=read-the-docs&logoColor=%23ffffff&color=%231BC070)](https://github.com/neuralmagic/guidellm/tree/main/docs) [![License](https://img.shields.io/github/license/neuralmagic/guidellm.svg)](https://github.com/neuralmagic/guidellm/blob/main/LICENSE) [![PyPI Release](https://img.shields.io/pypi/v/guidellm.svg?label=PyPI%20Release)](https://pypi.python.org/pypi/guidellm) [![Python Versions](https://img.shields.io/badge/Python-3.9--3.13-orange)](https://pypi.python.org/pypi/guidellm) [![Nightly Build](https://img.shields.io/github/actions/workflow/status/neuralmagic/guidellm/nightly.yml?branch=main&label=Nightly%20Build)](https://github.com/neuralmagic/guidellm/actions/workflows/nightly.yml)
 
 ## Overview
 
@@ -413,6 +413,7 @@ Our comprehensive documentation offers detailed guides and resources to help you
 
 - [**Installation Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/install.md) - This guide provides step-by-step instructions for installing GuideLLM, including prerequisites and setup tips.
 - [**Backends Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/backends.md) - A comprehensive overview of supported backends and how to set them up for use with GuideLLM.
+- [**Data/Datasets Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/datasets.md) - Information on supported datasets, including how to use them for benchmarking.
 - [**Metrics Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/metrics.md) - Detailed explanations of the metrics used in GuideLLM, including definitions and how to interpret them.
 - [**Outputs Guide**](https://github.com/neuralmagic/guidellm/tree/main/docs/outputs.md) - Information on the different output formats supported by GuideLLM and how to use them.
 - [**Architecture Overview**](https://github.com/neuralmagic/guidellm/tree/main/docs/architecture.md) - A detailed look at GuideLLM's design, components, and how they interact.