agentx-python 0.8.9__tar.gz → 0.8.11__tar.gz

{agentx_python-0.8.9/agentx_python.egg-info → agentx_python-0.8.11}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentx-python
-Version: 0.8.9
+Version: 0.8.11
 Summary: Official Python SDK for AgentX (https://www.agentx.so/)
 Home-page: https://github.com/AgentX-ai/AgentX-python
 Author: Robin Wang and AgentX Team
@@ -68,7 +68,7 @@ Dynamic: summary
 [![Python versions](https://img.shields.io/pypi/pyversions/agentx-python)](https://pypi.org/project/agentx-python/)
 [![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
 
-The official Python SDK for **[AgentX](https://app.agentx.so/)** - an evaluation, tracing, and monitoring framework for AI agents, plus a client for AgentX's own hosted agents.
+The official Python SDK for **[AgentX](https://www.agentx.so/)** - an evaluation, tracing, and monitoring framework for AI agents, plus a client for AgentX's own hosted agents.
 
 Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs](https://docs.agentx.so/reference)
 
@@ -109,19 +109,10 @@ pip install --upgrade agentx-python
 
 Requires Python 3.9 or newer.
 
----
-
-## Authentication
-
-Get your API key at [app.agentx.so](https://app.agentx.so), then either pass it inline or expose it as an environment variable.
-
-```python
-# Option A - pass the key inline
-from agentx import AgentX
-client = AgentX(api_key="your-api-key-here")
+#### Run self host eval framework locally
 
-# Option B - set AGENTX_API_KEY in your environment, then:
-client = AgentX.from_env()
+```
+agentx-trace-eval --dev --update
 ```
 
 ---
@@ -179,7 +170,7 @@ print(report.recommendations)      # list of prioritized, actionable fixes
 
 Ask a case's question several extra ways each run, LLM-paraphrased server-side, to catch agents that break on phrasing rather than substance, and override the judge's prompt/model per config, see [Smoke testing](EVALUATIONS.md#smoke-testing-phrasing-robustness) and [Configuring the judge](EVALUATIONS.md#configuring-the-judge) in the full guide.
 
-Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's *source of truth* instead - the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results - a human always has to approve before it publishes:
+Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's _source of truth_ instead - the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results - a human always has to approve before it publishes:
 
 ```python
 prompt = client.evaluations.prompts.get("support-agent-system-prompt")  # or prompt.id
@@ -193,7 +184,7 @@ client.evaluations.run(
 
 See [Prompt registry](EVALUATIONS.md#prompt-registry) in the full guide, or [self-host's docs](https://docs.agentx.so/improve/prompt-management) for the "Suggest improvement" dashboard flow (self-host only - no hosted-SaaS equivalent yet).
 
-On self-host, a finalized run can also **gate a CI job**: `report.gate(fail_under=7, no_regression=True)` checks the run's average rating against an absolute floor and/or the dataset's previous run, prints per-check verdicts into the CI log, and returns an exit code - `sys.exit(gate.exit_code)` blocks the merge on regression. Recorded gates appear in the dashboard's CI Gates tab. See [self-host's CI docs](https://docs.agentx.so/integrations/self-host-ci) for the GitHub Actions recipe.
+On self-host, a finalized run can also **gate a CI job**: `run.gate(fail_under=7, no_regression=True)` (on the run context `.execute()` returns) checks the run's average rating against an absolute floor and/or the dataset's previous run, prints per-check verdicts into the CI log, and returns an exit code - `sys.exit(gate.exit_code)` blocks the merge on regression. Recorded gates appear in the dashboard's CI Gates tab. See [self-host's CI docs](https://docs.agentx.so/integrations/self-host-ci) for the GitHub Actions recipe.
 
 See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide - dataset builder, framework adapters, similarity metrics, smoke testing, judge configuration, prompt registry, and the complete API reference.
 
@@ -248,18 +239,18 @@ regular totals, no extra config needed. Self-host's cost estimate prices these s
 regular input token when you've set optional cache rates on that model. Install the matching
 extra:
 
-| Framework             | Install                                      | Integration              |
-| --------------------- | -------------------------------------------- | ------------------------ |
-| LangChain             | `pip install "agentx-python[langchain]"`     | `AgentXCallbackHandler`  |
-| CrewAI                | `pip install "agentx-python[crewai]"`        | `AgentXCrewObserver`     |
-| OpenAI Agents SDK     | `pip install "agentx-python[openai-agents]"` | `AgentXTracingProcessor` |
-| OpenAI (raw client)   | `pip install "agentx-python[openai]"`        | `patch_openai_client`    |
-| Anthropic             | `pip install "agentx-python[anthropic]"`     | `patch_anthropic_client` |
-| Google ADK            | `pip install "agentx-python[google-adk]"`    | `AgentXADKPlugin`        |
-| Google GenAI (Gemini) | `pip install "agentx-python[google-genai]"`  | `patch_genai_client`     |
-| LiteLLM               | `pip install "agentx-python[litellm]"`       | `AgentXLiteLLMLogger`    |
-| LlamaIndex             | `pip install "agentx-python[llamaindex]"`    | `AgentXLlamaIndexHandler`|
-| AutoGen                | `pip install "agentx-python[autogen]"`       | `AgentXAutoGenObserver`  |
+| Framework             | Install                                      | Integration               |
+| --------------------- | -------------------------------------------- | ------------------------- |
+| LangChain             | `pip install "agentx-python[langchain]"`     | `AgentXCallbackHandler`   |
+| CrewAI                | `pip install "agentx-python[crewai]"`        | `AgentXCrewObserver`      |
+| OpenAI Agents SDK     | `pip install "agentx-python[openai-agents]"` | `AgentXTracingProcessor`  |
+| OpenAI (raw client)   | `pip install "agentx-python[openai]"`        | `patch_openai_client`     |
+| Anthropic             | `pip install "agentx-python[anthropic]"`     | `patch_anthropic_client`  |
+| Google ADK            | `pip install "agentx-python[google-adk]"`    | `AgentXADKPlugin`         |
+| Google GenAI (Gemini) | `pip install "agentx-python[google-genai]"`  | `patch_genai_client`      |
+| LiteLLM               | `pip install "agentx-python[litellm]"`       | `AgentXLiteLLMLogger`     |
+| LlamaIndex            | `pip install "agentx-python[llamaindex]"`    | `AgentXLlamaIndexHandler` |
+| AutoGen               | `pip install "agentx-python[autogen]"`       | `AgentXAutoGenObserver`   |
 
 Or plain Python - wrap any function with `@tracer.trace(...)` and it just works, no framework required.
 

{agentx_python-0.8.9 → agentx_python-0.8.11}/README.md

@@ -4,7 +4,7 @@
 [![Python versions](https://img.shields.io/pypi/pyversions/agentx-python)](https://pypi.org/project/agentx-python/)
 [![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
 
-The official Python SDK for **[AgentX](https://app.agentx.so/)** - an evaluation, tracing, and monitoring framework for AI agents, plus a client for AgentX's own hosted agents.
+The official Python SDK for **[AgentX](https://www.agentx.so/)** - an evaluation, tracing, and monitoring framework for AI agents, plus a client for AgentX's own hosted agents.
 
 Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs](https://docs.agentx.so/reference)
 
@@ -45,19 +45,10 @@ pip install --upgrade agentx-python
 
 Requires Python 3.9 or newer.
 
----
-
-## Authentication
-
-Get your API key at [app.agentx.so](https://app.agentx.so), then either pass it inline or expose it as an environment variable.
-
-```python
-# Option A - pass the key inline
-from agentx import AgentX
-client = AgentX(api_key="your-api-key-here")
+#### Run self host eval framework locally
 
-# Option B - set AGENTX_API_KEY in your environment, then:
-client = AgentX.from_env()
+```
+agentx-trace-eval --dev --update
 ```
 
 ---
@@ -115,7 +106,7 @@ print(report.recommendations)      # list of prioritized, actionable fixes
 
 Ask a case's question several extra ways each run, LLM-paraphrased server-side, to catch agents that break on phrasing rather than substance, and override the judge's prompt/model per config, see [Smoke testing](EVALUATIONS.md#smoke-testing-phrasing-robustness) and [Configuring the judge](EVALUATIONS.md#configuring-the-judge) in the full guide.
 
-Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's *source of truth* instead - the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results - a human always has to approve before it publishes:
+Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's _source of truth_ instead - the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results - a human always has to approve before it publishes:
 
 ```python
 prompt = client.evaluations.prompts.get("support-agent-system-prompt")  # or prompt.id
@@ -129,7 +120,7 @@ client.evaluations.run(
 
 See [Prompt registry](EVALUATIONS.md#prompt-registry) in the full guide, or [self-host's docs](https://docs.agentx.so/improve/prompt-management) for the "Suggest improvement" dashboard flow (self-host only - no hosted-SaaS equivalent yet).
 
-On self-host, a finalized run can also **gate a CI job**: `report.gate(fail_under=7, no_regression=True)` checks the run's average rating against an absolute floor and/or the dataset's previous run, prints per-check verdicts into the CI log, and returns an exit code - `sys.exit(gate.exit_code)` blocks the merge on regression. Recorded gates appear in the dashboard's CI Gates tab. See [self-host's CI docs](https://docs.agentx.so/integrations/self-host-ci) for the GitHub Actions recipe.
+On self-host, a finalized run can also **gate a CI job**: `run.gate(fail_under=7, no_regression=True)` (on the run context `.execute()` returns) checks the run's average rating against an absolute floor and/or the dataset's previous run, prints per-check verdicts into the CI log, and returns an exit code - `sys.exit(gate.exit_code)` blocks the merge on regression. Recorded gates appear in the dashboard's CI Gates tab. See [self-host's CI docs](https://docs.agentx.so/integrations/self-host-ci) for the GitHub Actions recipe.
 
 See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide - dataset builder, framework adapters, similarity metrics, smoke testing, judge configuration, prompt registry, and the complete API reference.
 
@@ -184,18 +175,18 @@ regular totals, no extra config needed. Self-host's cost estimate prices these s
 regular input token when you've set optional cache rates on that model. Install the matching
 extra:
 
-| Framework             | Install                                      | Integration              |
-| --------------------- | -------------------------------------------- | ------------------------ |
-| LangChain             | `pip install "agentx-python[langchain]"`     | `AgentXCallbackHandler`  |
-| CrewAI                | `pip install "agentx-python[crewai]"`        | `AgentXCrewObserver`     |
-| OpenAI Agents SDK     | `pip install "agentx-python[openai-agents]"` | `AgentXTracingProcessor` |
-| OpenAI (raw client)   | `pip install "agentx-python[openai]"`        | `patch_openai_client`    |
-| Anthropic             | `pip install "agentx-python[anthropic]"`     | `patch_anthropic_client` |
-| Google ADK            | `pip install "agentx-python[google-adk]"`    | `AgentXADKPlugin`        |
-| Google GenAI (Gemini) | `pip install "agentx-python[google-genai]"`  | `patch_genai_client`     |
-| LiteLLM               | `pip install "agentx-python[litellm]"`       | `AgentXLiteLLMLogger`    |
-| LlamaIndex             | `pip install "agentx-python[llamaindex]"`    | `AgentXLlamaIndexHandler`|
-| AutoGen                | `pip install "agentx-python[autogen]"`       | `AgentXAutoGenObserver`  |
+| Framework             | Install                                      | Integration               |
+| --------------------- | -------------------------------------------- | ------------------------- |
+| LangChain             | `pip install "agentx-python[langchain]"`     | `AgentXCallbackHandler`   |
+| CrewAI                | `pip install "agentx-python[crewai]"`        | `AgentXCrewObserver`      |
+| OpenAI Agents SDK     | `pip install "agentx-python[openai-agents]"` | `AgentXTracingProcessor`  |
+| OpenAI (raw client)   | `pip install "agentx-python[openai]"`        | `patch_openai_client`     |
+| Anthropic             | `pip install "agentx-python[anthropic]"`     | `patch_anthropic_client`  |
+| Google ADK            | `pip install "agentx-python[google-adk]"`    | `AgentXADKPlugin`         |
+| Google GenAI (Gemini) | `pip install "agentx-python[google-genai]"`  | `patch_genai_client`      |
+| LiteLLM               | `pip install "agentx-python[litellm]"`       | `AgentXLiteLLMLogger`     |
+| LlamaIndex            | `pip install "agentx-python[llamaindex]"`    | `AgentXLlamaIndexHandler` |
+| AutoGen               | `pip install "agentx-python[autogen]"`       | `AgentXAutoGenObserver`   |
 
 Or plain Python - wrap any function with `@tracer.trace(...)` and it just works, no framework required.
 

{agentx_python-0.8.9 → agentx_python-0.8.11}/agentx/evaluations/client.py

@@ -37,6 +37,9 @@ _RETRY_BACKOFF = [1.0, 2.0, 4.0]
 # wait out the whole job on one connection. Matches EvaluationRunContext.analyze()'s own
 # default timeout.
 _SELF_HOST_ANALYZE_TIMEOUT = 1800
+# Batch result submission scores each result synchronously inside the request (one judge call
+# per result on the sync path) - a big batch on a slow judge legitimately takes minutes.
+_SELF_HOST_SCORING_TIMEOUT = 900
 
 
 class AgentXEvaluationsError(Exception):
@@ -344,7 +347,15 @@ class EvaluationsClient:
             "batchId": batch_id,
             "results": [_result_to_payload(r) for r in results],
         }
-        data = self._request("POST", f"/runs/{run_id}/results", json=payload)
+        # Scoring is synchronous inside this request (a judge call per result - the runner's
+        # spinner says "~60s+" for a reason), so the default 30s timeout + silent backoff loop
+        # re-POSTed the batch WHILE the first submission was still scoring, and the two raced
+        # into the engine's (run_id, idempotency_key) unique constraint. Long timeout, no
+        # transport retry - the runner's own logged retry-once is the retry layer here.
+        data = self._request(
+            "POST", f"/runs/{run_id}/results", json=payload,
+            timeout=_SELF_HOST_SCORING_TIMEOUT, retry=False,
+        )
         return BatchAppendResponse(**data)
 
     def finalize_run(self, run_id: str) -> Dict[str, Any]:

{agentx_python-0.8.9 → agentx_python-0.8.11}/agentx/version.py

@@ -1,7 +1,7 @@
-VERSION = "0.8.9"
+VERSION = "0.8.11"
 
 # The AgentX-trace-eval release this SDK version is tested against - what `agentx-trace-eval`
 # installs and converges to (see agentx/cli.py). Bump together with VERSION when releasing, so
 # every published SDK names a known-good engine+dashboard pair. Users can override with
 # AGENTX_TRACE_EVAL_VERSION=<tag|latest>.
-ENGINE_VERSION = "v0.3.1"
+ENGINE_VERSION = "v0.3.2"

{agentx_python-0.8.9 → agentx_python-0.8.11/agentx_python.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentx-python
-Version: 0.8.9
+Version: 0.8.11
 Summary: Official Python SDK for AgentX (https://www.agentx.so/)
 Home-page: https://github.com/AgentX-ai/AgentX-python
 Author: Robin Wang and AgentX Team
@@ -68,7 +68,7 @@ Dynamic: summary
 [![Python versions](https://img.shields.io/pypi/pyversions/agentx-python)](https://pypi.org/project/agentx-python/)
 [![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
 
-The official Python SDK for **[AgentX](https://app.agentx.so/)** - an evaluation, tracing, and monitoring framework for AI agents, plus a client for AgentX's own hosted agents.
+The official Python SDK for **[AgentX](https://www.agentx.so/)** - an evaluation, tracing, and monitoring framework for AI agents, plus a client for AgentX's own hosted agents.
 
 Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs](https://docs.agentx.so/reference)
 
@@ -109,19 +109,10 @@ pip install --upgrade agentx-python
 
 Requires Python 3.9 or newer.
 
----
-
-## Authentication
-
-Get your API key at [app.agentx.so](https://app.agentx.so), then either pass it inline or expose it as an environment variable.
-
-```python
-# Option A - pass the key inline
-from agentx import AgentX
-client = AgentX(api_key="your-api-key-here")
+#### Run self host eval framework locally
 
-# Option B - set AGENTX_API_KEY in your environment, then:
-client = AgentX.from_env()
+```
+agentx-trace-eval --dev --update
 ```
 
 ---
@@ -179,7 +170,7 @@ print(report.recommendations)      # list of prioritized, actionable fixes
 
 Ask a case's question several extra ways each run, LLM-paraphrased server-side, to catch agents that break on phrasing rather than substance, and override the judge's prompt/model per config, see [Smoke testing](EVALUATIONS.md#smoke-testing-phrasing-robustness) and [Configuring the judge](EVALUATIONS.md#configuring-the-judge) in the full guide.
 
-Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's *source of truth* instead - the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results - a human always has to approve before it publishes:
+Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's _source of truth_ instead - the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results - a human always has to approve before it publishes:
 
 ```python
 prompt = client.evaluations.prompts.get("support-agent-system-prompt")  # or prompt.id
@@ -193,7 +184,7 @@ client.evaluations.run(
 
 See [Prompt registry](EVALUATIONS.md#prompt-registry) in the full guide, or [self-host's docs](https://docs.agentx.so/improve/prompt-management) for the "Suggest improvement" dashboard flow (self-host only - no hosted-SaaS equivalent yet).
 
-On self-host, a finalized run can also **gate a CI job**: `report.gate(fail_under=7, no_regression=True)` checks the run's average rating against an absolute floor and/or the dataset's previous run, prints per-check verdicts into the CI log, and returns an exit code - `sys.exit(gate.exit_code)` blocks the merge on regression. Recorded gates appear in the dashboard's CI Gates tab. See [self-host's CI docs](https://docs.agentx.so/integrations/self-host-ci) for the GitHub Actions recipe.
+On self-host, a finalized run can also **gate a CI job**: `run.gate(fail_under=7, no_regression=True)` (on the run context `.execute()` returns) checks the run's average rating against an absolute floor and/or the dataset's previous run, prints per-check verdicts into the CI log, and returns an exit code - `sys.exit(gate.exit_code)` blocks the merge on regression. Recorded gates appear in the dashboard's CI Gates tab. See [self-host's CI docs](https://docs.agentx.so/integrations/self-host-ci) for the GitHub Actions recipe.
 
 See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide - dataset builder, framework adapters, similarity metrics, smoke testing, judge configuration, prompt registry, and the complete API reference.
 
@@ -248,18 +239,18 @@ regular totals, no extra config needed. Self-host's cost estimate prices these s
 regular input token when you've set optional cache rates on that model. Install the matching
 extra:
 
-| Framework             | Install                                      | Integration              |
-| --------------------- | -------------------------------------------- | ------------------------ |
-| LangChain             | `pip install "agentx-python[langchain]"`     | `AgentXCallbackHandler`  |
-| CrewAI                | `pip install "agentx-python[crewai]"`        | `AgentXCrewObserver`     |
-| OpenAI Agents SDK     | `pip install "agentx-python[openai-agents]"` | `AgentXTracingProcessor` |
-| OpenAI (raw client)   | `pip install "agentx-python[openai]"`        | `patch_openai_client`    |
-| Anthropic             | `pip install "agentx-python[anthropic]"`     | `patch_anthropic_client` |
-| Google ADK            | `pip install "agentx-python[google-adk]"`    | `AgentXADKPlugin`        |
-| Google GenAI (Gemini) | `pip install "agentx-python[google-genai]"`  | `patch_genai_client`     |
-| LiteLLM               | `pip install "agentx-python[litellm]"`       | `AgentXLiteLLMLogger`    |
-| LlamaIndex             | `pip install "agentx-python[llamaindex]"`    | `AgentXLlamaIndexHandler`|
-| AutoGen                | `pip install "agentx-python[autogen]"`       | `AgentXAutoGenObserver`  |
+| Framework             | Install                                      | Integration               |
+| --------------------- | -------------------------------------------- | ------------------------- |
+| LangChain             | `pip install "agentx-python[langchain]"`     | `AgentXCallbackHandler`   |
+| CrewAI                | `pip install "agentx-python[crewai]"`        | `AgentXCrewObserver`      |
+| OpenAI Agents SDK     | `pip install "agentx-python[openai-agents]"` | `AgentXTracingProcessor`  |
+| OpenAI (raw client)   | `pip install "agentx-python[openai]"`        | `patch_openai_client`     |
+| Anthropic             | `pip install "agentx-python[anthropic]"`     | `patch_anthropic_client`  |
+| Google ADK            | `pip install "agentx-python[google-adk]"`    | `AgentXADKPlugin`         |
+| Google GenAI (Gemini) | `pip install "agentx-python[google-genai]"`  | `patch_genai_client`      |
+| LiteLLM               | `pip install "agentx-python[litellm]"`       | `AgentXLiteLLMLogger`     |
+| LlamaIndex            | `pip install "agentx-python[llamaindex]"`    | `AgentXLlamaIndexHandler` |
+| AutoGen               | `pip install "agentx-python[autogen]"`       | `AgentXAutoGenObserver`   |
 
 Or plain Python - wrap any function with `@tracer.trace(...)` and it just works, no framework required.