agenta 0.57.1__tar.gz → 0.60.0__tar.gz

{agenta-0.57.1 → agenta-0.60.0}/PKG-INFO

@@ -1,29 +1,29 @@
 Metadata-Version: 2.4
 Name: agenta
-Version: 0.57.1
+Version: 0.60.0
 Summary: The SDK for agenta is an open-source LLMOps platform.
 Keywords: LLMOps,LLM,evaluation,prompt engineering
 Author: Mahmoud Mabrouk
 Author-email: mahmoud@agenta.ai
-Requires-Python: >=3.9,<4.0
+Requires-Python: >=3.11,<4.0
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.9
 Classifier: Topic :: Software Development :: Libraries
 Requires-Dist: decorator (>=5.2.1,<6.0.0)
 Requires-Dist: fastapi (>=0.116.0,<0.117.0)
+Requires-Dist: google-auth (>=2.23,<3)
 Requires-Dist: h11 (>=0.16.0,<0.17.0)
 Requires-Dist: httpx (>=0.28.0,<0.29.0)
 Requires-Dist: huggingface-hub (<0.31.0)
 Requires-Dist: importlib-metadata (>=8.0.0,<9.0)
 Requires-Dist: jinja2 (>=3.1.6,<4.0.0)
-Requires-Dist: litellm (==1.76.0)
+Requires-Dist: litellm (==1.78.7)
 Requires-Dist: openai (>=1.106.0,<2.0.0)
 Requires-Dist: opentelemetry-api (>=1.27.0,<2.0.0)
 Requires-Dist: opentelemetry-exporter-otlp-proto-http (>=1.27.0,<2.0.0)
@@ -31,7 +31,9 @@ Requires-Dist: opentelemetry-instrumentation (>=0.56b0)
 Requires-Dist: opentelemetry-sdk (>=1.27.0,<2.0.0)
 Requires-Dist: pydantic (>=2,<3)
 Requires-Dist: python-dotenv (>=1.0.0,<2.0.0)
+Requires-Dist: python-jsonpath (>=2.0.0,<3.0.0)
 Requires-Dist: pyyaml (>=6.0.2,<7.0.0)
+Requires-Dist: restrictedpython (>=8.0,<9.0) ; python_version == "3.11"
 Requires-Dist: starlette (>=0.47.0,<0.48.0)
 Requires-Dist: structlog (>=25.2.0,<26.0.0)
 Requires-Dist: tiktoken (==0.11.0)
@@ -126,15 +128,15 @@ Agenta is a platform for building production-grade LLM applications. It helps **
 Collaborate with Subject Matter Experts (SMEs) on prompt engineering and make sure nothing breaks in production.
 
 - **Interactive Playground**: Compare prompts side by side against your test cases
-- **Multi-Model Support**: Experiment with 50+ LLM models or [bring-your-own models](https://docs.agenta.ai/prompt-engineering/playground/adding-custom-providers?utm_source=github&utm_medium=referral&utm_campaign=readme)
+- **Multi-Model Support**: Experiment with 50+ LLM models or [bring-your-own models](https://docs.agenta.ai/prompt-engineering/playground/custom-providers?utm_source=github&utm_medium=referral&utm_campaign=readme)
 - **Version Control**: Version prompts and configurations with branching and environments
 - **Complex Configurations**: Enable SMEs to collaborate on [complex configuration schemas](https://docs.agenta.ai/custom-workflows/overview?utm_source=github&utm_medium=referral&utm_campaign=readme) beyond simple prompts
 
-[Explore prompt management →](https://docs.agenta.ai/prompt-engineering/overview?utm_source=github&utm_medium=referral&utm_campaign=readme)
+[Explore prompt management →](https://docs.agenta.ai/prompt-engineering/concepts?utm_source=github&utm_medium=referral&utm_campaign=readme)
 
 ### 📊 Evaluation & Testing
 Evaluate your LLM applications systematically with both human and automated feedback.
-- **Flexible Test Sets**: Create test cases from production data, playground experiments, or upload CSVs
+- **Flexible Testsets**: Create testcases from production data, playground experiments, or upload CSVs
 - **Pre-built and Custom Evaluators**: Use LLM-as-judge, one of our 20+ pre-built evaluators, or you custom evaluators
 - **UI and API Access**: Run evaluations via UI (for SMEs) or programmatically (for engineers)
 - **Human Feedback Integration**: Collect and incorporate expert annotations

{agenta-0.57.1 → agenta-0.60.0}/README.md

@@ -83,15 +83,15 @@ Agenta is a platform for building production-grade LLM applications. It helps **
 Collaborate with Subject Matter Experts (SMEs) on prompt engineering and make sure nothing breaks in production.
 
 - **Interactive Playground**: Compare prompts side by side against your test cases
-- **Multi-Model Support**: Experiment with 50+ LLM models or [bring-your-own models](https://docs.agenta.ai/prompt-engineering/playground/adding-custom-providers?utm_source=github&utm_medium=referral&utm_campaign=readme)
+- **Multi-Model Support**: Experiment with 50+ LLM models or [bring-your-own models](https://docs.agenta.ai/prompt-engineering/playground/custom-providers?utm_source=github&utm_medium=referral&utm_campaign=readme)
 - **Version Control**: Version prompts and configurations with branching and environments
 - **Complex Configurations**: Enable SMEs to collaborate on [complex configuration schemas](https://docs.agenta.ai/custom-workflows/overview?utm_source=github&utm_medium=referral&utm_campaign=readme) beyond simple prompts
 
-[Explore prompt management →](https://docs.agenta.ai/prompt-engineering/overview?utm_source=github&utm_medium=referral&utm_campaign=readme)
+[Explore prompt management →](https://docs.agenta.ai/prompt-engineering/concepts?utm_source=github&utm_medium=referral&utm_campaign=readme)
 
 ### 📊 Evaluation & Testing
 Evaluate your LLM applications systematically with both human and automated feedback.
-- **Flexible Test Sets**: Create test cases from production data, playground experiments, or upload CSVs
+- **Flexible Testsets**: Create testcases from production data, playground experiments, or upload CSVs
 - **Pre-built and Custom Evaluators**: Use LLM-as-judge, one of our 20+ pre-built evaluators, or you custom evaluators
 - **UI and API Access**: Run evaluations via UI (for SMEs) or programmatically (for engineers)
 - **Human Feedback Integration**: Collect and incorporate expert annotations

{agenta-0.57.1 → agenta-0.60.0}/agenta/__init__.py

@@ -27,8 +27,12 @@ from .sdk.utils.costs import calculate_token_usage
 from .sdk.tracing import Tracing, get_tracer
 from .sdk.tracing.conventions import Reference
 from .sdk.decorators.tracing import instrument
-from .sdk.decorators.running import workflow, workflows
-from .sdk.decorators.serving import entrypoint, app, route
+from .sdk.decorators.running import (
+    workflow,
+    application,
+    evaluator,
+)
+from .sdk.decorators.serving import route, app
 from .sdk.context.running import workflow_mode_enabled
 from .sdk.litellm import litellm as callbacks
 from .sdk.managers.apps import AppManager

{agenta-0.57.1 → agenta-0.60.0}/agenta/client/__init__.py

@@ -160,8 +160,8 @@ from .backend.types import (
     StandardProviderSettingsDto,
     StatusCode,
     StatusDto,
-    TestSetOutputResponse,
-    TestSetSimpleResponse,
+    TestsetOutputResponse,
+    TestsetSimpleResponse,
     TestcaseResponse,
     Testset,
     TestsetRequest,
@@ -403,8 +403,8 @@ __all__ = [
     "StandardProviderSettingsDto",
     "StatusCode",
     "StatusDto",
-    "TestSetOutputResponse",
-    "TestSetSimpleResponse",
+    "TestsetOutputResponse",
+    "TestsetSimpleResponse",
     "TestcaseResponse",
     "Testset",
     "TestsetRequest",

{agenta-0.57.1 → agenta-0.60.0}/agenta/client/backend/__init__.py

@@ -155,8 +155,8 @@ from .types import (
     StandardProviderSettingsDto,
     StatusCode,
     StatusDto,
-    TestSetOutputResponse,
-    TestSetSimpleResponse,
+    TestsetOutputResponse,
+    TestsetSimpleResponse,
     TestcaseResponse,
     Testset,
     TestsetRequest,
@@ -398,8 +398,8 @@ __all__ = [
     "StandardProviderSettingsDto",
     "StatusCode",
     "StatusDto",
-    "TestSetOutputResponse",
-    "TestSetSimpleResponse",
+    "TestsetOutputResponse",
+    "TestsetSimpleResponse",
     "TestcaseResponse",
     "Testset",
     "TestsetRequest",

{agenta-0.57.1 → agenta-0.60.0}/agenta/client/backend/client.py

@@ -98,13 +98,17 @@ class AgentaApi:
         self._client_wrapper = SyncClientWrapper(
             base_url=_get_base_url(base_url=base_url, environment=environment),
             api_key=api_key,
-            httpx_client=httpx_client
-            if httpx_client is not None
-            else httpx.Client(
-                timeout=_defaulted_timeout, follow_redirects=follow_redirects
-            )
-            if follow_redirects is not None
-            else httpx.Client(timeout=_defaulted_timeout),
+            httpx_client=(
+                httpx_client
+                if httpx_client is not None
+                else (
+                    httpx.Client(
+                        timeout=_defaulted_timeout, follow_redirects=follow_redirects
+                    )
+                    if follow_redirects is not None
+                    else httpx.Client(timeout=_defaulted_timeout)
+                )
+            ),
             timeout=_defaulted_timeout,
         )
         self._raw_client = RawAgentaApi(client_wrapper=self._client_wrapper)
@@ -172,7 +176,7 @@ class AgentaApi:
 
     def update_organization(
         self,
-        org_id: str,
+        organization_id: str,
         *,
         name: typing.Optional[str] = OMIT,
         description: typing.Optional[str] = OMIT,
@@ -182,7 +186,7 @@ class AgentaApi:
         """
         Parameters
         ----------
-        org_id : str
+        organization_id : str
 
         name : typing.Optional[str]
 
@@ -206,11 +210,11 @@ class AgentaApi:
             api_key="YOUR_API_KEY",
         )
         client.update_organization(
-            org_id="org_id",
+            organization_id="organization_id",
         )
         """
         _response = self._raw_client.update_organization(
-            org_id,
+            organization_id,
             name=name,
             description=description,
             updated_at=updated_at,
@@ -220,7 +224,7 @@ class AgentaApi:
 
     def create_workspace(
         self,
-        org_id: str,
+        organization_id: str,
         *,
         name: str,
         description: typing.Optional[str] = OMIT,
@@ -230,7 +234,7 @@ class AgentaApi:
         """
         Parameters
         ----------
-        org_id : str
+        organization_id : str
 
         name : str
 
@@ -254,12 +258,12 @@ class AgentaApi:
             api_key="YOUR_API_KEY",
         )
         client.create_workspace(
-            org_id="org_id",
+            organization_id="organization_id",
             name="name",
         )
         """
         _response = self._raw_client.create_workspace(
-            org_id,
+            organization_id,
             name=name,
             description=description,
             type=type,
@@ -269,7 +273,7 @@ class AgentaApi:
 
     def update_workspace(
         self,
-        org_id: str,
+        organization_id: str,
         workspace_id: str,
         *,
         name: typing.Optional[str] = OMIT,
@@ -280,7 +284,7 @@ class AgentaApi:
         """
         Parameters
         ----------
-        org_id : str
+        organization_id : str
 
         workspace_id : str
 
@@ -306,12 +310,12 @@ class AgentaApi:
             api_key="YOUR_API_KEY",
         )
         client.update_workspace(
-            org_id="org_id",
+            organization_id="organization_id",
             workspace_id="workspace_id",
         )
         """
         _response = self._raw_client.update_workspace(
-            org_id,
+            organization_id,
             workspace_id,
             name=name,
             description=description,
@@ -427,7 +431,7 @@ class AgentaApi:
         workspace_id: str,
         *,
         email: str,
-        org_id: str,
+        organization_id: str,
         role: str,
         request_options: typing.Optional[RequestOptions] = None,
     ) -> typing.Optional[typing.Any]:
@@ -437,7 +441,7 @@ class AgentaApi:
         Args:
             workspace_id (str): The ID of the workspace.
             email (str): The email of the user to remove the role from.
-            org_id (str): The ID of the organization.
+            organization_id (str): The ID of the organization.
             role (str): The role to remove from the user.
             request (Request): The FastAPI request object.
 
@@ -454,7 +458,7 @@ class AgentaApi:
 
         email : str
 
-        org_id : str
+        organization_id : str
 
         role : str
 
@@ -476,14 +480,14 @@ class AgentaApi:
         client.unassign_role_from_user(
             workspace_id="workspace_id",
             email="email",
-            org_id="org_id",
+            organization_id="organization_id",
             role="role",
         )
         """
         _response = self._raw_client.unassign_role_from_user(
             workspace_id,
             email=email,
-            org_id=org_id,
+            organization_id=organization_id,
             role=role,
             request_options=request_options,
         )
@@ -657,13 +661,17 @@ class AsyncAgentaApi:
         self._client_wrapper = AsyncClientWrapper(
             base_url=_get_base_url(base_url=base_url, environment=environment),
             api_key=api_key,
-            httpx_client=httpx_client
-            if httpx_client is not None
-            else httpx.AsyncClient(
-                timeout=_defaulted_timeout, follow_redirects=follow_redirects
-            )
-            if follow_redirects is not None
-            else httpx.AsyncClient(timeout=_defaulted_timeout),
+            httpx_client=(
+                httpx_client
+                if httpx_client is not None
+                else (
+                    httpx.AsyncClient(
+                        timeout=_defaulted_timeout, follow_redirects=follow_redirects
+                    )
+                    if follow_redirects is not None
+                    else httpx.AsyncClient(timeout=_defaulted_timeout)
+                )
+            ),
             timeout=_defaulted_timeout,
         )
         self._raw_client = AsyncRawAgentaApi(client_wrapper=self._client_wrapper)
@@ -743,7 +751,7 @@ class AsyncAgentaApi:
 
     async def update_organization(
         self,
-        org_id: str,
+        organization_id: str,
         *,
         name: typing.Optional[str] = OMIT,
         description: typing.Optional[str] = OMIT,
@@ -753,7 +761,7 @@ class AsyncAgentaApi:
         """
         Parameters
         ----------
-        org_id : str
+        organization_id : str
 
         name : typing.Optional[str]
 
@@ -782,14 +790,14 @@ class AsyncAgentaApi:
 
         async def main() -> None:
             await client.update_organization(
-                org_id="org_id",
+                organization_id="organization_id",
             )
 
 
         asyncio.run(main())
         """
         _response = await self._raw_client.update_organization(
-            org_id,
+            organization_id,
             name=name,
             description=description,
             updated_at=updated_at,
@@ -799,7 +807,7 @@ class AsyncAgentaApi:
 
     async def create_workspace(
         self,
-        org_id: str,
+        organization_id: str,
         *,
         name: str,
         description: typing.Optional[str] = OMIT,
@@ -809,7 +817,7 @@ class AsyncAgentaApi:
         """
         Parameters
         ----------
-        org_id : str
+        organization_id : str
 
         name : str
 
@@ -838,7 +846,7 @@ class AsyncAgentaApi:
 
         async def main() -> None:
             await client.create_workspace(
-                org_id="org_id",
+                organization_id="organization_id",
                 name="name",
             )
 
@@ -846,7 +854,7 @@ class AsyncAgentaApi:
         asyncio.run(main())
         """
         _response = await self._raw_client.create_workspace(
-            org_id,
+            organization_id,
             name=name,
             description=description,
             type=type,
@@ -856,7 +864,7 @@ class AsyncAgentaApi:
 
     async def update_workspace(
         self,
-        org_id: str,
+        organization_id: str,
         workspace_id: str,
         *,
         name: typing.Optional[str] = OMIT,
@@ -867,7 +875,7 @@ class AsyncAgentaApi:
         """
         Parameters
         ----------
-        org_id : str
+        organization_id : str
 
         workspace_id : str
 
@@ -898,7 +906,7 @@ class AsyncAgentaApi:
 
         async def main() -> None:
             await client.update_workspace(
-                org_id="org_id",
+                organization_id="organization_id",
                 workspace_id="workspace_id",
             )
 
@@ -906,7 +914,7 @@ class AsyncAgentaApi:
         asyncio.run(main())
         """
         _response = await self._raw_client.update_workspace(
-            org_id,
+            organization_id,
             workspace_id,
             name=name,
             description=description,
@@ -1038,7 +1046,7 @@ class AsyncAgentaApi:
         workspace_id: str,
         *,
         email: str,
-        org_id: str,
+        organization_id: str,
         role: str,
         request_options: typing.Optional[RequestOptions] = None,
     ) -> typing.Optional[typing.Any]:
@@ -1048,7 +1056,7 @@ class AsyncAgentaApi:
         Args:
             workspace_id (str): The ID of the workspace.
             email (str): The email of the user to remove the role from.
-            org_id (str): The ID of the organization.
+            organization_id (str): The ID of the organization.
             role (str): The role to remove from the user.
             request (Request): The FastAPI request object.
 
@@ -1065,7 +1073,7 @@ class AsyncAgentaApi:
 
         email : str
 
-        org_id : str
+        organization_id : str
 
         role : str
 
@@ -1092,7 +1100,7 @@ class AsyncAgentaApi:
             await client.unassign_role_from_user(
                 workspace_id="workspace_id",
                 email="email",
-                org_id="org_id",
+                organization_id="organization_id",
                 role="role",
             )
 
@@ -1102,7 +1110,7 @@ class AsyncAgentaApi:
         _response = await self._raw_client.unassign_role_from_user(
             workspace_id,
             email=email,
-            org_id=org_id,
+            organization_id=organization_id,
             role=role,
             request_options=request_options,
         )

{agenta-0.57.1 → agenta-0.60.0}/agenta/client/backend/human_evaluations/client.py

@@ -237,7 +237,7 @@ class HumanEvaluationsClient:
         Updates an evaluation's status.
 
         Raises:
-            HTTPException: If the columns in the test set do not match with the inputs in the variant.
+            HTTPException: If the columns in the testset do not match with the inputs in the variant.
 
         Returns:
             None: A 204 No Content status code, indicating that the update was successful.
@@ -785,7 +785,7 @@ class AsyncHumanEvaluationsClient:
         Updates an evaluation's status.
 
         Raises:
-            HTTPException: If the columns in the test set do not match with the inputs in the variant.
+            HTTPException: If the columns in the testset do not match with the inputs in the variant.
 
         Returns:
             None: A 204 No Content status code, indicating that the update was successful.

{agenta-0.57.1 → agenta-0.60.0}/agenta/client/backend/human_evaluations/raw_client.py

@@ -336,7 +336,7 @@ class RawHumanEvaluationsClient:
         Updates an evaluation's status.
 
         Raises:
-            HTTPException: If the columns in the test set do not match with the inputs in the variant.
+            HTTPException: If the columns in the testset do not match with the inputs in the variant.
 
         Returns:
             None: A 204 No Content status code, indicating that the update was successful.
@@ -1112,7 +1112,7 @@ class AsyncRawHumanEvaluationsClient:
         Updates an evaluation's status.
 
         Raises:
-            HTTPException: If the columns in the test set do not match with the inputs in the variant.
+            HTTPException: If the columns in the testset do not match with the inputs in the variant.
 
         Returns:
             None: A 204 No Content status code, indicating that the update was successful.