agentx-python 0.4.8__tar.gz → 0.4.10__tar.gz

agentx_python-0.4.10/PKG-INFO

@@ -0,0 +1,198 @@
+Metadata-Version: 2.4
+Name: agentx-python
+Version: 0.4.10
+Summary: Official Python SDK for AgentX (https://www.agentx.so/)
+Home-page: https://github.com/AgentX-ai/AgentX-python
+Author: Robin Wang and AgentX Team
+Author-email: contact@agentx.so
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: urllib3>=1.26.11
+Requires-Dist: certifi
+Requires-Dist: requests
+Requires-Dist: pydantic
+Requires-Dist: pydantic_core
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+![Logo](https://agentx-resources.s3.us-west-1.amazonaws.com/AgentX-logo-387x60.png)
+
+[![PyPI version](https://img.shields.io/pypi/v/agentx-python)](https://pypi.org/project/agentx-python/)
+
+The official Python SDK for **[AgentX](https://www.agentx.so/)** — build, chat with, and orchestrate AI agents in a few lines of code.
+
+---
+
+## Contents
+
+- [Why AgentX](#why-agentx)
+- [Installation](#installation)
+- [Authentication](#authentication)
+- [Quick start](#quick-start)
+- [Working with agents](#working-with-agents)
+  - [List agents](#list-agents)
+  - [Start a conversation](#start-a-conversation)
+  - [Chat (streaming and non-streaming)](#chat-streaming-and-non-streaming)
+- [Workforce (multi-agent orchestration)](#workforce-multi-agent-orchestration)
+- [Agent Evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity
+- [Links](#links)
+
+---
+
+## Why AgentX
+
+- **Simple mental model** — `Agent → Conversation → Message`.
+- **Chain-of-thought** is built in, no extra plumbing.
+- **Bring any LLM** — works across major open and closed-source vendors.
+- **Batteries included** — voice (ASR/TTS), image generation, document/CSV/Excel/OCR, RAG with built-in re-ranking.
+- **MCP support** — connect any Model Context Protocol server.
+- **Multi-agent orchestration** — workforces of agents with a designated manager, across LLM vendors.
+- **Agent Evaluations** — score any agent (LangChain, CrewAI, OpenAI, Anthropic, HTTP, …) with LLM-as-a-judge ratings plus optional cosine and Jaccard similarity metrics.
+- **A2A** — Each agent can be published with agent-to-agent protocol compatible.
+
+---
+
+## Installation
+
+```bash
+pip install --upgrade agentx-python
+```
+
+Requires Python 3.9 or newer.
+
+---
+
+## Authentication
+
+Get your API key at [app.agentx.so](https://app.agentx.so), then either pass it inline or expose it as an environment variable.
+
+```python
+# Option A — pass the key inline
+from agentx import AgentX
+client = AgentX(api_key="your-api-key-here")
+
+# Option B — set AGENTX_API_KEY in your environment, then:
+client = AgentX.from_env()
+```
+
+---
+
+## Quick start
+
+```python
+from agentx import AgentX
+
+client = AgentX.from_env()
+
+# Pick an existing agent and chat with it
+agent = client.list_agents()[0]
+conversation = agent.new_conversation()
+print(conversation.chat("Hello! What can you help me with?"))
+```
+
+That's it. The remaining sections show the same primitives in more detail.
+
+---
+
+## Working with agents
+
+### List agents
+
+```python
+agents = client.list_agents()
+print(f"You have {len(agents)} agents")
+```
+
+### Start a conversation
+
+```python
+agent = client.get_agent(id="<agent-id>")
+
+# Either resume an existing conversation…
+existing = agent.list_conversations()
+last = existing[-1]
+for msg in last.list_messages():
+    print(msg)
+
+# …or start a fresh one
+conversation = agent.new_conversation()
+```
+
+### Chat (streaming and non-streaming)
+
+```python
+# Blocking — returns the full response once it's ready
+response = conversation.chat("What is your name?")
+print(response)
+
+# Streaming — yields ChatResponse objects as the model produces them
+for chunk in conversation.chat_stream("Hello, what is your name?"):
+    if chunk.text:
+        print(chunk.text, end="")
+```
+
+Each `ChatResponse` chunk exposes the agent's `text` and, where applicable, its `cot` (chain-of-thought) reasoning, along with any retrieved references and tasks.
+
+---
+
+## Workforce (multi-agent orchestration)
+
+A **workforce** is a team of agents coordinated by a designated manager agent. Workforces can mix LLM vendors and route work between specialists.
+
+```python
+workforces = client.list_workforces()
+workforce = workforces[0]
+
+print(f"Workforce: {workforce.name}")
+print(f"Manager:   {workforce.manager.name}")
+print(f"Agents:    {[a.name for a in workforce.agents]}")
+
+# Chat with the workforce — the manager decides which agent(s) to delegate to
+conversation = workforce.new_conversation()
+for chunk in workforce.chat_stream(conversation.id, "How can you help me with this project?"):
+    if chunk.text:
+        print(chunk.text, end="")
+```
+
+---
+
+## Custom agent evaluations
+
+Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine** and **Jaccard** similarity metrics alongside LLM-graded ratings.
+
+```python
+report = (
+    client.evaluations
+    .run(dataset_id="evds_…", subject={"kind": "custom_agent", "framework": "raw_python"})
+    .execute(my_agent_fn)
+    .finalize()
+    .analyze()
+)
+
+print(report.average_rating)       # LLM-graded score, 0–10
+print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
+print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
+```
+
+See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, and the complete API reference.
+
+---
+
+## Links
+
+- **Dashboard** — [app.agentx.so](https://app.agentx.so)
+- **Website** — [agentx.so](https://www.agentx.so/)
+- **PyPI** — [agentx-python](https://pypi.org/project/agentx-python/)
+- **Evaluations docs** — [EVALUATIONS.md](EVALUATIONS.md)

agentx_python-0.4.10/README.md

@@ -0,0 +1,169 @@
+![Logo](https://agentx-resources.s3.us-west-1.amazonaws.com/AgentX-logo-387x60.png)
+
+[![PyPI version](https://img.shields.io/pypi/v/agentx-python)](https://pypi.org/project/agentx-python/)
+
+The official Python SDK for **[AgentX](https://www.agentx.so/)** — build, chat with, and orchestrate AI agents in a few lines of code.
+
+---
+
+## Contents
+
+- [Why AgentX](#why-agentx)
+- [Installation](#installation)
+- [Authentication](#authentication)
+- [Quick start](#quick-start)
+- [Working with agents](#working-with-agents)
+  - [List agents](#list-agents)
+  - [Start a conversation](#start-a-conversation)
+  - [Chat (streaming and non-streaming)](#chat-streaming-and-non-streaming)
+- [Workforce (multi-agent orchestration)](#workforce-multi-agent-orchestration)
+- [Agent Evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity
+- [Links](#links)
+
+---
+
+## Why AgentX
+
+- **Simple mental model** — `Agent → Conversation → Message`.
+- **Chain-of-thought** is built in, no extra plumbing.
+- **Bring any LLM** — works across major open and closed-source vendors.
+- **Batteries included** — voice (ASR/TTS), image generation, document/CSV/Excel/OCR, RAG with built-in re-ranking.
+- **MCP support** — connect any Model Context Protocol server.
+- **Multi-agent orchestration** — workforces of agents with a designated manager, across LLM vendors.
+- **Agent Evaluations** — score any agent (LangChain, CrewAI, OpenAI, Anthropic, HTTP, …) with LLM-as-a-judge ratings plus optional cosine and Jaccard similarity metrics.
+- **A2A** — Each agent can be published with agent-to-agent protocol compatible.
+
+---
+
+## Installation
+
+```bash
+pip install --upgrade agentx-python
+```
+
+Requires Python 3.9 or newer.
+
+---
+
+## Authentication
+
+Get your API key at [app.agentx.so](https://app.agentx.so), then either pass it inline or expose it as an environment variable.
+
+```python
+# Option A — pass the key inline
+from agentx import AgentX
+client = AgentX(api_key="your-api-key-here")
+
+# Option B — set AGENTX_API_KEY in your environment, then:
+client = AgentX.from_env()
+```
+
+---
+
+## Quick start
+
+```python
+from agentx import AgentX
+
+client = AgentX.from_env()
+
+# Pick an existing agent and chat with it
+agent = client.list_agents()[0]
+conversation = agent.new_conversation()
+print(conversation.chat("Hello! What can you help me with?"))
+```
+
+That's it. The remaining sections show the same primitives in more detail.
+
+---
+
+## Working with agents
+
+### List agents
+
+```python
+agents = client.list_agents()
+print(f"You have {len(agents)} agents")
+```
+
+### Start a conversation
+
+```python
+agent = client.get_agent(id="<agent-id>")
+
+# Either resume an existing conversation…
+existing = agent.list_conversations()
+last = existing[-1]
+for msg in last.list_messages():
+    print(msg)
+
+# …or start a fresh one
+conversation = agent.new_conversation()
+```
+
+### Chat (streaming and non-streaming)
+
+```python
+# Blocking — returns the full response once it's ready
+response = conversation.chat("What is your name?")
+print(response)
+
+# Streaming — yields ChatResponse objects as the model produces them
+for chunk in conversation.chat_stream("Hello, what is your name?"):
+    if chunk.text:
+        print(chunk.text, end="")
+```
+
+Each `ChatResponse` chunk exposes the agent's `text` and, where applicable, its `cot` (chain-of-thought) reasoning, along with any retrieved references and tasks.
+
+---
+
+## Workforce (multi-agent orchestration)
+
+A **workforce** is a team of agents coordinated by a designated manager agent. Workforces can mix LLM vendors and route work between specialists.
+
+```python
+workforces = client.list_workforces()
+workforce = workforces[0]
+
+print(f"Workforce: {workforce.name}")
+print(f"Manager:   {workforce.manager.name}")
+print(f"Agents:    {[a.name for a in workforce.agents]}")
+
+# Chat with the workforce — the manager decides which agent(s) to delegate to
+conversation = workforce.new_conversation()
+for chunk in workforce.chat_stream(conversation.id, "How can you help me with this project?"):
+    if chunk.text:
+        print(chunk.text, end="")
+```
+
+---
+
+## Custom agent evaluations
+
+Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine** and **Jaccard** similarity metrics alongside LLM-graded ratings.
+
+```python
+report = (
+    client.evaluations
+    .run(dataset_id="evds_…", subject={"kind": "custom_agent", "framework": "raw_python"})
+    .execute(my_agent_fn)
+    .finalize()
+    .analyze()
+)
+
+print(report.average_rating)       # LLM-graded score, 0–10
+print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
+print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
+```
+
+See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, and the complete API reference.
+
+---
+
+## Links
+
+- **Dashboard** — [app.agentx.so](https://app.agentx.so)
+- **Website** — [agentx.so](https://www.agentx.so/)
+- **PyPI** — [agentx-python](https://pypi.org/project/agentx-python/)
+- **Evaluations docs** — [EVALUATIONS.md](EVALUATIONS.md)

{agentx_python-0.4.8 → agentx_python-0.4.10}/agentx/agentx.py

@@ -3,37 +3,51 @@ import requests
 import os
 import logging
 
-from agentx.util import get_headers
+from agentx.util import get_headers, api_base
 from agentx.resources.agent import Agent
 from agentx.resources.workforce import Workforce
 
 
 class AgentX:
 
-    def __init__(self, api_key: str = None):
+    def __init__(self, api_key: str = None, base_url: str = None):
         self.api_key = api_key or os.getenv("AGENTX_API_KEY")
         if self.api_key and not os.getenv("AGENTX_API_KEY"):
             os.environ["AGENTX_API_KEY"] = self.api_key
 
+        # base_url overrides AGENTX_API_BASE_URL env var (and the SDK default)
+        self.base_url = base_url or os.getenv("AGENTX_API_BASE_URL")
+        if self.base_url:
+            os.environ["AGENTX_API_BASE_URL"] = self.base_url
+
+        from agentx.evaluations.client import EvaluationsClient
+        from agentx.evaluations.runner import EvaluationsRunner
+        from agentx.version import VERSION
+
+        _eval_client = EvaluationsClient(
+            api_key=self.api_key,
+            sdk_version=VERSION,
+            base_url=self.base_url,
+        )
+        self.evaluations = EvaluationsRunner(_eval_client)
+
+    @classmethod
+    def from_env(cls) -> "AgentX":
+        """Create an AgentX client using AGENTX_API_KEY (and optionally AGENTX_API_BASE_URL) from the environment."""
+        return cls()
+
     def get_agent(self, id: str) -> Agent:
-        url = f"https://api.agentx.so/api/v1/access/agents/{id}"
+        url = f"{api_base()}/access/agents/{id}"
         # Make a GET request to the AgentX API
         response = requests.get(url, headers=get_headers())
         # Check if response was successful
         if response.status_code == 200:
-            agent_res = response.json()
-            return Agent(
-                id=agent_res.get("_id"),
-                name=agent_res.get("name"),
-                avatar=agent_res.get("avatar"),
-                createdAt=agent_res.get("createdAt"),
-                updatedAt=agent_res.get("updatedAt"),
-            )
+            return Agent(**response.json())
         else:
             raise Exception(f"Failed to retrieve agent: {response.reason}")
 
     def list_agents(self) -> List[Agent]:
-        url = "https://api.agentx.so/api/v1/access/agents"
+        url = f"{api_base()}/access/agents"
         # Make a GET request to the AgentX API
         response = requests.get(url, headers=get_headers())
         # Check if response was successful
@@ -45,7 +59,7 @@ class AgentX:
     @staticmethod
     def list_workforces() -> List["Workforce"]:
         """List all workforces/teams."""
-        url = "https://api.agentx.so/api/v1/access/teams"
+        url = f"{api_base()}/access/teams"
         response = requests.get(url, headers=get_headers())
         if response.status_code == 200:
             return [Workforce(**workforce) for workforce in response.json()]
@@ -56,7 +70,7 @@ class AgentX:
 
     def get_profile(self):
         """Get the current user's profile information."""
-        url = "https://api.agentx.so/api/v1/access/getProfile"
+        url = f"{api_base()}/access/getProfile"
         response = requests.get(url, headers=get_headers())
         if response.status_code == 200:
             return response.json()

agentx_python-0.4.10/agentx/evaluations/__init__.py

@@ -0,0 +1,3 @@
+from agentx.evaluations.client import EvaluationsClient
+
+__all__ = ["EvaluationsClient"]

agentx_python-0.4.10/agentx/evaluations/_term.py

@@ -0,0 +1,91 @@
+"""Minimal ANSI terminal helpers — no external dependencies."""
+
+from __future__ import annotations
+
+import itertools
+import sys
+import threading
+import time
+
+_IS_TTY = hasattr(sys.stdout, "isatty") and sys.stdout.isatty()
+
+
+def _c(code: str) -> str:
+    return f"\033[{code}m" if _IS_TTY else ""
+
+
+RESET = _c("0")
+BOLD = _c("1")
+DIM = _c("2")
+GREEN = _c("32")
+YELLOW = _c("33")
+RED = _c("31")
+CYAN = _c("36")
+MAGENTA = _c("35")
+
+
+def green(s: str) -> str:
+    return f"{GREEN}{s}{RESET}"
+
+
+def yellow(s: str) -> str:
+    return f"{YELLOW}{s}{RESET}"
+
+
+def red(s: str) -> str:
+    return f"{RED}{s}{RESET}"
+
+
+def cyan(s: str) -> str:
+    return f"{CYAN}{s}{RESET}"
+
+
+def bold(s: str) -> str:
+    return f"{BOLD}{s}{RESET}"
+
+
+def dim(s: str) -> str:
+    return f"{DIM}{s}{RESET}"
+
+
+def magenta(s: str) -> str:
+    return f"{MAGENTA}{s}{RESET}"
+
+
+class Spinner:
+    """Inline spinner that overwrites the current line on a TTY."""
+
+    _FRAMES = ("⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏")
+
+    def __init__(self, message: str):
+        self._message = message
+        self._stop = threading.Event()
+        self._thread: threading.Thread | None = None
+
+    def __enter__(self) -> "Spinner":
+        if not _IS_TTY:
+            print(f"  {self._message}...", flush=True)
+            return self
+        self._stop.clear()
+        self._thread = threading.Thread(target=self._spin, daemon=True)
+        self._thread.start()
+        return self
+
+    def __exit__(self, *_) -> None:
+        if not _IS_TTY:
+            return
+        self._stop.set()
+        if self._thread:
+            self._thread.join()
+        sys.stdout.write(f"\r{' ' * (len(self._message) + 12)}\r")
+        sys.stdout.flush()
+
+    def _spin(self) -> None:
+        for frame in itertools.cycle(self._FRAMES):
+            if self._stop.is_set():
+                break
+            sys.stdout.write(
+                f"\r  {CYAN}{frame}{RESET}  {self._message}  {DIM}(this may take ~60s+){RESET}"
+            )
+            sys.stdout.flush()
+            time.sleep(0.08)

agentx_python-0.4.10/agentx/evaluations/adapters/__init__.py

@@ -0,0 +1,5 @@
+from agentx.evaluations.adapters.raw import RawCallableAdapter
+from agentx.evaluations.adapters.precomputed import PrecomputedAdapter
+from agentx.evaluations.adapters.http_endpoint import HttpEndpointAdapter
+
+__all__ = ["RawCallableAdapter", "PrecomputedAdapter", "HttpEndpointAdapter"]

agentx_python-0.4.10/agentx/evaluations/adapters/http_endpoint.py

@@ -0,0 +1,69 @@
+from __future__ import annotations
+
+import time
+from typing import Any, Dict, Optional
+
+import requests
+
+from agentx.evaluations.models import EvaluationCase, EvaluationResult
+from agentx.evaluations.results import normalize_error, normalize_result
+
+
+class HttpEndpointAdapter:
+    """
+    Calls a user-hosted HTTP endpoint for each evaluation case.
+    The SDK (running locally) makes the request — the AgentX API never
+    touches the customer's endpoint.
+
+    The endpoint receives a POST with::
+
+        {"query": "...", "case_id": "...", "metadata": {...}}
+
+    It should respond with a JSON body containing at least ``output``
+    (or ``text``) and optionally ``trace`` and ``metadata``.
+
+    Usage::
+
+        adapter = HttpEndpointAdapter(
+            url="http://localhost:8080/eval",
+            headers={"Authorization": "Bearer my-token"},
+            timeout=30,
+        )
+    """
+
+    def __init__(
+        self,
+        url: str,
+        headers: Optional[Dict[str, str]] = None,
+        timeout: int = 30,
+        method: str = "POST",
+    ):
+        self._url = url
+        self._headers = headers or {}
+        self._timeout = timeout
+        self._method = method.upper()
+
+    def run(self, case: EvaluationCase) -> EvaluationResult:
+        payload = {
+            "query": case.query,
+            "case_id": case.case_id,
+            "question_index": case.question_index,
+            "run_number": case.run_number,
+        }
+        start = time.monotonic()
+        try:
+            resp = requests.request(
+                self._method,
+                self._url,
+                json=payload,
+                headers=self._headers,
+                timeout=self._timeout,
+            )
+            resp.raise_for_status()
+            elapsed = int((time.monotonic() - start) * 1000)
+            raw = resp.json()
+        except Exception as exc:
+            elapsed = int((time.monotonic() - start) * 1000)
+            return normalize_error(case, exc, latency_ms=elapsed)
+
+        return normalize_result(case, raw, latency_ms=elapsed)

agentx_python-0.4.10/agentx/evaluations/adapters/precomputed.py

@@ -0,0 +1,35 @@
+from __future__ import annotations
+
+from typing import Any, Dict, List, Union
+
+from agentx.evaluations.models import EvaluationCase, EvaluationResult
+from agentx.evaluations.results import normalize_result
+
+
+class PrecomputedAdapter:
+    """
+    Adapter for pre-computed outputs — useful when you already have agent
+    responses and just want AgentX to score them.
+
+    Accepts a list or dict keyed by case_id::
+
+        outputs = {
+            "case-0": "You can reset your password from account settings.",
+            "case-1": {"output": "Contact support@example.com", "metadata": {...}},
+        }
+        adapter = PrecomputedAdapter(outputs)
+    """
+
+    def __init__(self, outputs: Union[List[Any], Dict[str, Any]]):
+        if isinstance(outputs, list):
+            self._lookup: Dict[str, Any] = {str(i): v for i, v in enumerate(outputs)}
+        else:
+            self._lookup = {str(k): v for k, v in outputs.items()}
+
+    def run(self, case: EvaluationCase) -> EvaluationResult:
+        raw = self._lookup.get(case.case_id) or self._lookup.get(
+            str(case.question_index)
+        )
+        if raw is None:
+            raw = ""
+        return normalize_result(case, raw, latency_ms=0)

agentx_python-0.4.10/agentx/evaluations/adapters/raw.py

@@ -0,0 +1,38 @@
+from __future__ import annotations
+
+from typing import Any, Callable
+
+from agentx.evaluations.models import EvaluationCase, EvaluationResult
+from agentx.evaluations.results import normalize_error, normalize_result
+
+
+class RawCallableAdapter:
+    """
+    Wraps any Python callable that accepts an EvaluationCase and returns
+    str | dict | EvaluationResult.
+
+    Usage::
+
+        def my_agent(case: EvaluationCase) -> str:
+            return my_llm.invoke(case.query)
+
+        adapter = RawCallableAdapter(my_agent)
+        result = adapter.run(case)
+    """
+
+    def __init__(self, fn: Callable[[EvaluationCase], Any]):
+        self._fn = fn
+
+    def run(
+        self, case: EvaluationCase, latency_ms: int | None = None
+    ) -> EvaluationResult:
+        import time
+
+        start = time.monotonic()
+        try:
+            raw = self._fn(case)
+        except Exception as exc:
+            elapsed = int((time.monotonic() - start) * 1000)
+            return normalize_error(case, exc, latency_ms=elapsed)
+        elapsed = int((time.monotonic() - start) * 1000)
+        return normalize_result(case, raw, latency_ms=elapsed)