agentx-python 0.6.13__tar.gz → 0.6.15__tar.gz

{agentx_python-0.6.13/agentx_python.egg-info → agentx_python-0.6.15}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentx-python
-Version: 0.6.13
+Version: 0.6.15
 Summary: Official Python SDK for AgentX (https://www.agentx.so/)
 Home-page: https://github.com/AgentX-ai/AgentX-python
 Author: Robin Wang and AgentX Team
@@ -66,7 +66,7 @@ Dynamic: summary
 [![Python versions](https://img.shields.io/pypi/pyversions/agentx-python)](https://pypi.org/project/agentx-python/)
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
 
-The official Python SDK for **[AgentX](https://app.agentx.so/)** — build, chat with, orchestrate, and trace AI agents in a few lines of code.
+The official Python SDK for **[AgentX](https://app.agentx.so/)** — an evaluation, tracing, and monitoring framework for AI agents, plus a client for AgentX's own hosted agents.
 
 Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs](https://docs.agentx.so/reference)
 
@@ -78,30 +78,24 @@ Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs
 - [Installation](#installation)
 - [Authentication](#authentication)
 - [Quick start](#quick-start)
-- [Working with agents](#working-with-agents)
-  - [List agents](#list-agents)
-  - [Start a conversation](#start-a-conversation)
-  - [Chat (streaming and non-streaming)](#chat-streaming-and-non-streaming)
-- [Workforce (multi-agent orchestration)](#workforce-multi-agent-orchestration) — teams of agents with a designated manager
+- [Custom agent evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity, any framework
 - [Production tracing](#production-tracing) — record live agent runs from any framework
 - [Monitor](#monitor) — automatic production monitoring, patterns and signals
-- [Custom agent evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity
 - [Self-host](#self-host) — run Trace/Evaluate/Monitor on your own machine instead of the hosted dashboard
+- [Agents & conversations](#agents--conversations) — chat with and orchestrate AgentX's own hosted agents
 - [Links](#links)
 
 ---
 
 ## Why AgentX
 
-- **Simple mental model** — `Agent → Conversation → Message`.
-- **Chain-of-thought** is built in, no extra plumbing.
-- **Bring any LLM** — works across major open and closed-source vendors.
-- **Batteries included** — voice (ASR/TTS), image generation, document/CSV/Excel/OCR, RAG with built-in re-ranking.
-- **MCP support** — connect any Model Context Protocol server.
-- **Multi-agent orchestration** — workforces of agents with a designated manager, across LLM vendors.
-- **Production tracing** — one decorator or context manager records every agent run (input, output, latency, tool calls, token usage) into your workspace, for any framework.
-- **Agent Evaluations** — score any agent (LangChain, CrewAI, OpenAI, Anthropic, HTTP, …) with LLM-as-a-judge ratings plus optional cosine and Jaccard similarity metrics. Configurable judge prompt/model (OpenAI or Anthropic), per-question judge guidelines, and smoke testing for phrasing robustness.
-- **A2A** — Each agent can be published with agent-to-agent protocol compatible.
+- **Agent Evaluations** — score **any** agent (LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP, or plain Python) against a dataset with LLM-as-a-judge ratings plus optional cosine, Jaccard, and BLEU/ROUGE similarity metrics. Configurable judge prompt/model, per-question judge guidelines, smoke testing for phrasing robustness, and a durable multi-judge analysis pass for a qualitative report.
+- **Production tracing** — one decorator or context manager records every agent run (input, output, latency, tool calls, token usage, and a real span tree) into your workspace, for any framework.
+- **Monitor** — check live traces against detection patterns or a sampled LLM judge, and read back triage-ready signals, no dashboard setup required.
+- **Prompt registry** — make AgentX the source of truth for your own agent's prompts: pull a version at runtime, tag eval runs and live traces with it, let a judge propose a rewrite from your worst-rated results.
+- **Self-host** — run the whole Trace/Evaluate/Monitor stack locally, bring your own LLM keys, no account required.
+- **Bring any LLM** — works across major open and closed-source vendors, for evaluation, tracing, and AgentX's own hosted agents alike.
+- **AgentX's own hosted agents** — a simple `Agent → Conversation → Message` mental model, chain-of-thought built in, multi-agent workforces, MCP support, and A2A publishing, for when you want AgentX to run the agent too, not just evaluate/trace/monitor it.
 
 ---
 
@@ -132,80 +126,72 @@ client = AgentX.from_env()
 
 ## Quick start
 
+Evaluate your own agent — any framework, or plain Python — against a dataset:
+
 ```python
 from agentx import AgentX
 
 client = AgentX.from_env()
 
-# Pick an existing agent and chat with it
-agent = client.list_agents()[0]
-conversation = agent.new_conversation()
-print(conversation.chat("Hello! What can you help me with?"))
+def my_agent(case):
+    return call_my_agent(case.query)  # your agent's own code, any framework
+
+report = (
+    client.evaluations
+    .run(dataset_id="evds_…", subject={"kind": "custom_agent", "framework": "raw_python"})
+    .execute(my_agent)
+    .finalize()
+    .analyze()
+)
+
+print(report.average_rating)   # LLM-graded score, 0–10
+print(report.summary)          # AI-generated narrative from .analyze()
 ```
 
-That's it. The remaining sections show the same primitives in more detail.
+That's it. The rest of this section covers building the dataset, framework adapters, similarity metrics, and judge configuration.
 
 ---
 
-## Working with agents
+## Custom agent evaluations
 
-### List agents
+Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine**, **Jaccard**, and **BLEU/ROUGE** similarity metrics alongside LLM-graded ratings.
 
 ```python
-agents = client.list_agents()
-print(f"You have {len(agents)} agents")
-```
+report = (
+    client.evaluations
+    .run(dataset_id="evds_…", subject={"kind": "custom_agent", "framework": "raw_python"})
+    .execute(my_agent_fn)
+    .finalize()
+    .analyze()
+)
 
-### Start a conversation
+print(report.average_rating)       # LLM-graded score, 0–10
+print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
+print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
 
-```python
-agent = client.get_agent(id="<agent-id>")
+print(report.summary)              # AI-generated narrative from .analyze()
+print(report.recommendations)      # list of prioritized, actionable fixes
+```
 
-# Either resume an existing conversation…
-existing = agent.list_conversations()
-last = existing[-1]
-for msg in last.list_messages():
-    print(msg)
+`.analyze()` also generates a full qualitative report (strengths, weaknesses, instruction adherence, reasoning quality, and recommendations), running the same durable, multi-judge pipeline as the dashboard's "Analyze" button. `analyze(mode=..., quality_mode=..., judges=[...])` controls how items are scored and by which models. See [AI analysis report](EVALUATIONS.md#ai-analysis-report) in the full guide for the complete field and parameter reference.
 
-# …or start a fresh one
-conversation = agent.new_conversation()
-```
+Ask a case's question several extra ways each run, LLM-paraphrased server-side, to catch agents that break on phrasing rather than substance, and override the judge's prompt/model per config, see [Smoke testing](EVALUATIONS.md#smoke-testing-phrasing-robustness) and [Configuring the judge](EVALUATIONS.md#configuring-the-judge) in the full guide.
 
-### Chat (streaming and non-streaming)
+Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's *source of truth* instead — the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results — a human always has to approve before it publishes:
 
 ```python
-# Blocking — returns the full response once it's ready
-response = conversation.chat("What is your name?")
-print(response)
+prompt = client.evaluations.prompts.get("support-agent-system-prompt")  # or prompt.id
+# use prompt.text as your own agent's system prompt
 
-# Streaming — yields ChatResponse objects as the model produces them
-for chunk in conversation.chat_stream("Hello, what is your name?"):
-    if chunk.text:
-        print(chunk.text, end="")
+client.evaluations.run(
+    dataset_id="evds_…",
+    subject={"kind": "custom_agent", "metadata": {"promptName": prompt.name}},
+).execute(my_agent_fn)
 ```
 
-Each `ChatResponse` chunk exposes the agent's `text` and, where applicable, its `cot` (chain-of-thought) reasoning, along with any retrieved references and tasks.
-
----
-
-## Workforce (multi-agent orchestration)
-
-A **workforce** is a team of agents coordinated by a designated manager agent. Workforces can mix LLM vendors and route work between specialists.
-
-```python
-workforces = client.list_workforces()
-workforce = workforces[0]
-
-print(f"Workforce: {workforce.name}")
-print(f"Manager:   {workforce.manager.name}")
-print(f"Agents:    {[a.name for a in workforce.agents]}")
+See [Prompt registry](EVALUATIONS.md#prompt-registry) in the full guide, or [self-host's docs](https://docs.agentx.so/self-host#prompt-registry) for the "Suggest improvement" dashboard flow (self-host only — no hosted-SaaS equivalent yet).
 
-# Chat with the workforce — the manager decides which agent(s) to delegate to
-conversation = workforce.new_conversation()
-for chunk in workforce.chat_stream(conversation.id, "How can you help me with this project?"):
-    if chunk.text:
-        print(chunk.text, end="")
-```
+See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, smoke testing, judge configuration, prompt registry, and the complete API reference.
 
 ---
 
@@ -242,7 +228,12 @@ with tracer.trace("rag-agent", framework="langchain") as span:
 
 ### Framework integrations
 
-Each integration auto-captures LLM calls, tool calls, and token usage — install the matching extra:
+Each integration auto-captures LLM calls, tool calls, and token usage — including prompt-caching
+token counts (Anthropic's cache write/read, OpenAI/LiteLLM's cached tokens, Google GenAI's cached
+content), reported as their own `cache_read_tokens`/`cache_write_tokens` fields alongside the
+regular totals, no extra config needed. Self-host's cost estimate prices these separately from a
+regular input token when you've set optional cache rates on that model. Install the matching
+extra:
 
 | Framework             | Install                                      | Integration              |
 | --------------------- | -------------------------------------------- | ------------------------ |
@@ -315,66 +306,55 @@ See **[TRACING.md](TRACING.md)** for the complete Monitor guide.
 
 ---
 
-## Custom agent evaluations
-
-Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine** and **Jaccard** similarity metrics alongside LLM-graded ratings.
-
-```python
-report = (
-    client.evaluations
-    .run(dataset_id="evds_…", subject={"kind": "custom_agent", "framework": "raw_python"})
-    .execute(my_agent_fn)
-    .finalize()
-    .analyze()
-)
+## Self-host
 
-print(report.average_rating)       # LLM-graded score, 0–10
-print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
-print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
+Prefer to run Trace/Evaluate/Monitor locally instead of the hosted dashboard — no account, bring your own LLM keys? This SDK ships a launcher for [AgentX-trace-eval](https://github.com/AgentX-ai/AgentX-trace-eval), a separate, portable governance engine:
 
-print(report.summary)              # AI-generated narrative from .analyze()
-print(report.recommendations)      # list of prioritized, actionable fixes
+```bash
+agentx-trace-eval --dev
 ```
 
-`.analyze()` also generates a full qualitative report (strengths, weaknesses, instruction adherence, reasoning quality, and recommendations), running the same durable, multi-judge pipeline as the dashboard's "Analyze" button. `analyze(mode=..., quality_mode=..., judges=[...])` controls how items are scored and by which models. See [AI analysis report](EVALUATIONS.md#ai-analysis-report) in the full guide for the complete field and parameter reference.
+The first run downloads the engine (and dashboard) into `~/.agentx/bin` and prints a local API key; every run after that just starts it. Point this SDK at it instead of the hosted API:
 
-Ask a case's question several extra ways each run, LLM-paraphrased server-side, to catch agents that break on phrasing rather than substance, and override the judge's prompt/model per config, see [Smoke testing](EVALUATIONS.md#smoke-testing-phrasing-robustness) and [Configuring the judge](EVALUATIONS.md#configuring-the-judge) in the full guide.
+```bash
+export AGENTX_API_BASE_URL=http://localhost:4700/api/v1
+export AGENTX_API_KEY=<printed by agentx-trace-eval on first run>
+```
 
-Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's *source of truth* instead — the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results — a human always has to approve before it publishes:
+`agentx-trace-eval` isn't this SDK's own code — the engine itself is a separate, compiled binary, downloaded on demand rather than bundled into this package, so installing `agentx-python` doesn't get any heavier for the (much more common) case of just talking to the hosted AgentX API. See that repo's README for what's included, and `AGENTX_INSTALL_DIR`/`AGENTX_TRACE_EVAL_VERSION`/`AGENTX_TRACE_EVAL_SKIP_WEB` env vars to control where/what it installs.
 
-```python
-prompt = client.evaluations.prompts.get("support-agent-system-prompt")  # or prompt.id
-# use prompt.text as your own agent's system prompt
+---
 
-client.evaluations.run(
-    dataset_id="evds_…",
-    subject={"kind": "custom_agent", "metadata": {"promptName": prompt.name}},
-).execute(my_agent_fn)
-```
+## Agents & conversations
 
-See [Prompt registry](EVALUATIONS.md#prompt-registry) in the full guide, or [self-host's docs](https://docs.agentx.so/self-host#prompt-registry) for the "Suggest improvement" dashboard flow (self-host only — no hosted-SaaS equivalent yet).
+Beyond evaluation, tracing, and monitoring, this SDK is also a client for AgentX's own hosted agents — build, chat with, and orchestrate them directly.
 
-See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, smoke testing, judge configuration, prompt registry, and the complete API reference.
+```python
+agent = client.list_agents()[0]
+conversation = agent.new_conversation()
 
----
+# Blocking — returns the full response once it's ready
+print(conversation.chat("What can you help me with?"))
 
-## Self-host
+# Streaming — yields ChatResponse objects as the model produces them
+for chunk in conversation.chat_stream("Hello!"):
+    if chunk.text:
+        print(chunk.text, end="")
+```
 
-Prefer to run Trace/Evaluate/Monitor locally instead of the hosted dashboard — no account, bring your own LLM keys? This SDK ships a launcher for [AgentX-trace-eval](https://github.com/AgentX-ai/AgentX-trace-eval), a separate, portable governance engine:
+Each `ChatResponse` chunk exposes the agent's `text` and, where applicable, its `cot` (chain-of-thought) reasoning, along with any retrieved references and tasks. `agent.list_conversations()` / `conversation.list_messages()` resume history instead of starting fresh.
 
-```bash
-agentx-trace-eval --dev
-```
+A **workforce** is a team of agents coordinated by a designated manager, mixing LLM vendors and routing work between specialists:
 
-The first run downloads the engine (and dashboard) into `~/.agentx/bin` and prints a local API key; every run after that just starts it. Point this SDK at it instead of the hosted API:
+```python
+workforce = client.list_workforces()[0]
+conversation = workforce.new_conversation()
 
-```bash
-export AGENTX_API_BASE_URL=http://localhost:4700/api/v1
-export AGENTX_API_KEY=<printed by agentx-trace-eval on first run>
+for chunk in workforce.chat_stream(conversation.id, "How can you help me with this project?"):
+    if chunk.text:
+        print(chunk.text, end="")
 ```
 
-`agentx-trace-eval` isn't this SDK's own code — the engine itself is a separate, compiled binary, downloaded on demand rather than bundled into this package, so installing `agentx-python` doesn't get any heavier for the (much more common) case of just talking to the hosted AgentX API. See that repo's README for what's included, and `AGENTX_INSTALL_DIR`/`AGENTX_TRACE_EVAL_VERSION`/`AGENTX_TRACE_EVAL_SKIP_WEB` env vars to control where/what it installs.
-
 ---
 
 ## Links

{agentx_python-0.6.13 → agentx_python-0.6.15}/README.md

@@ -4,7 +4,7 @@
 [![Python versions](https://img.shields.io/pypi/pyversions/agentx-python)](https://pypi.org/project/agentx-python/)
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
 
-The official Python SDK for **[AgentX](https://app.agentx.so/)** — build, chat with, orchestrate, and trace AI agents in a few lines of code.
+The official Python SDK for **[AgentX](https://app.agentx.so/)** — an evaluation, tracing, and monitoring framework for AI agents, plus a client for AgentX's own hosted agents.
 
 Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs](https://docs.agentx.so/reference)
 
@@ -16,30 +16,24 @@ Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs
 - [Installation](#installation)
 - [Authentication](#authentication)
 - [Quick start](#quick-start)
-- [Working with agents](#working-with-agents)
-  - [List agents](#list-agents)
-  - [Start a conversation](#start-a-conversation)
-  - [Chat (streaming and non-streaming)](#chat-streaming-and-non-streaming)
-- [Workforce (multi-agent orchestration)](#workforce-multi-agent-orchestration) — teams of agents with a designated manager
+- [Custom agent evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity, any framework
 - [Production tracing](#production-tracing) — record live agent runs from any framework
 - [Monitor](#monitor) — automatic production monitoring, patterns and signals
-- [Custom agent evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity
 - [Self-host](#self-host) — run Trace/Evaluate/Monitor on your own machine instead of the hosted dashboard
+- [Agents & conversations](#agents--conversations) — chat with and orchestrate AgentX's own hosted agents
 - [Links](#links)
 
 ---
 
 ## Why AgentX
 
-- **Simple mental model** — `Agent → Conversation → Message`.
-- **Chain-of-thought** is built in, no extra plumbing.
-- **Bring any LLM** — works across major open and closed-source vendors.
-- **Batteries included** — voice (ASR/TTS), image generation, document/CSV/Excel/OCR, RAG with built-in re-ranking.
-- **MCP support** — connect any Model Context Protocol server.
-- **Multi-agent orchestration** — workforces of agents with a designated manager, across LLM vendors.
-- **Production tracing** — one decorator or context manager records every agent run (input, output, latency, tool calls, token usage) into your workspace, for any framework.
-- **Agent Evaluations** — score any agent (LangChain, CrewAI, OpenAI, Anthropic, HTTP, …) with LLM-as-a-judge ratings plus optional cosine and Jaccard similarity metrics. Configurable judge prompt/model (OpenAI or Anthropic), per-question judge guidelines, and smoke testing for phrasing robustness.
-- **A2A** — Each agent can be published with agent-to-agent protocol compatible.
+- **Agent Evaluations** — score **any** agent (LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP, or plain Python) against a dataset with LLM-as-a-judge ratings plus optional cosine, Jaccard, and BLEU/ROUGE similarity metrics. Configurable judge prompt/model, per-question judge guidelines, smoke testing for phrasing robustness, and a durable multi-judge analysis pass for a qualitative report.
+- **Production tracing** — one decorator or context manager records every agent run (input, output, latency, tool calls, token usage, and a real span tree) into your workspace, for any framework.
+- **Monitor** — check live traces against detection patterns or a sampled LLM judge, and read back triage-ready signals, no dashboard setup required.
+- **Prompt registry** — make AgentX the source of truth for your own agent's prompts: pull a version at runtime, tag eval runs and live traces with it, let a judge propose a rewrite from your worst-rated results.
+- **Self-host** — run the whole Trace/Evaluate/Monitor stack locally, bring your own LLM keys, no account required.
+- **Bring any LLM** — works across major open and closed-source vendors, for evaluation, tracing, and AgentX's own hosted agents alike.
+- **AgentX's own hosted agents** — a simple `Agent → Conversation → Message` mental model, chain-of-thought built in, multi-agent workforces, MCP support, and A2A publishing, for when you want AgentX to run the agent too, not just evaluate/trace/monitor it.
 
 ---
 
@@ -70,80 +64,72 @@ client = AgentX.from_env()
 
 ## Quick start
 
+Evaluate your own agent — any framework, or plain Python — against a dataset:
+
 ```python
 from agentx import AgentX
 
 client = AgentX.from_env()
 
-# Pick an existing agent and chat with it
-agent = client.list_agents()[0]
-conversation = agent.new_conversation()
-print(conversation.chat("Hello! What can you help me with?"))
+def my_agent(case):
+    return call_my_agent(case.query)  # your agent's own code, any framework
+
+report = (
+    client.evaluations
+    .run(dataset_id="evds_…", subject={"kind": "custom_agent", "framework": "raw_python"})
+    .execute(my_agent)
+    .finalize()
+    .analyze()
+)
+
+print(report.average_rating)   # LLM-graded score, 0–10
+print(report.summary)          # AI-generated narrative from .analyze()
 ```
 
-That's it. The remaining sections show the same primitives in more detail.
+That's it. The rest of this section covers building the dataset, framework adapters, similarity metrics, and judge configuration.
 
 ---
 
-## Working with agents
+## Custom agent evaluations
 
-### List agents
+Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine**, **Jaccard**, and **BLEU/ROUGE** similarity metrics alongside LLM-graded ratings.
 
 ```python
-agents = client.list_agents()
-print(f"You have {len(agents)} agents")
-```
+report = (
+    client.evaluations
+    .run(dataset_id="evds_…", subject={"kind": "custom_agent", "framework": "raw_python"})
+    .execute(my_agent_fn)
+    .finalize()
+    .analyze()
+)
 
-### Start a conversation
+print(report.average_rating)       # LLM-graded score, 0–10
+print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
+print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
 
-```python
-agent = client.get_agent(id="<agent-id>")
+print(report.summary)              # AI-generated narrative from .analyze()
+print(report.recommendations)      # list of prioritized, actionable fixes
+```
 
-# Either resume an existing conversation…
-existing = agent.list_conversations()
-last = existing[-1]
-for msg in last.list_messages():
-    print(msg)
+`.analyze()` also generates a full qualitative report (strengths, weaknesses, instruction adherence, reasoning quality, and recommendations), running the same durable, multi-judge pipeline as the dashboard's "Analyze" button. `analyze(mode=..., quality_mode=..., judges=[...])` controls how items are scored and by which models. See [AI analysis report](EVALUATIONS.md#ai-analysis-report) in the full guide for the complete field and parameter reference.
 
-# …or start a fresh one
-conversation = agent.new_conversation()
-```
+Ask a case's question several extra ways each run, LLM-paraphrased server-side, to catch agents that break on phrasing rather than substance, and override the judge's prompt/model per config, see [Smoke testing](EVALUATIONS.md#smoke-testing-phrasing-robustness) and [Configuring the judge](EVALUATIONS.md#configuring-the-judge) in the full guide.
 
-### Chat (streaming and non-streaming)
+Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's *source of truth* instead — the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results — a human always has to approve before it publishes:
 
 ```python
-# Blocking — returns the full response once it's ready
-response = conversation.chat("What is your name?")
-print(response)
+prompt = client.evaluations.prompts.get("support-agent-system-prompt")  # or prompt.id
+# use prompt.text as your own agent's system prompt
 
-# Streaming — yields ChatResponse objects as the model produces them
-for chunk in conversation.chat_stream("Hello, what is your name?"):
-    if chunk.text:
-        print(chunk.text, end="")
+client.evaluations.run(
+    dataset_id="evds_…",
+    subject={"kind": "custom_agent", "metadata": {"promptName": prompt.name}},
+).execute(my_agent_fn)
 ```
 
-Each `ChatResponse` chunk exposes the agent's `text` and, where applicable, its `cot` (chain-of-thought) reasoning, along with any retrieved references and tasks.
-
----
-
-## Workforce (multi-agent orchestration)
-
-A **workforce** is a team of agents coordinated by a designated manager agent. Workforces can mix LLM vendors and route work between specialists.
-
-```python
-workforces = client.list_workforces()
-workforce = workforces[0]
-
-print(f"Workforce: {workforce.name}")
-print(f"Manager:   {workforce.manager.name}")
-print(f"Agents:    {[a.name for a in workforce.agents]}")
+See [Prompt registry](EVALUATIONS.md#prompt-registry) in the full guide, or [self-host's docs](https://docs.agentx.so/self-host#prompt-registry) for the "Suggest improvement" dashboard flow (self-host only — no hosted-SaaS equivalent yet).
 
-# Chat with the workforce — the manager decides which agent(s) to delegate to
-conversation = workforce.new_conversation()
-for chunk in workforce.chat_stream(conversation.id, "How can you help me with this project?"):
-    if chunk.text:
-        print(chunk.text, end="")
-```
+See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, smoke testing, judge configuration, prompt registry, and the complete API reference.
 
 ---
 
@@ -180,7 +166,12 @@ with tracer.trace("rag-agent", framework="langchain") as span:
 
 ### Framework integrations
 
-Each integration auto-captures LLM calls, tool calls, and token usage — install the matching extra:
+Each integration auto-captures LLM calls, tool calls, and token usage — including prompt-caching
+token counts (Anthropic's cache write/read, OpenAI/LiteLLM's cached tokens, Google GenAI's cached
+content), reported as their own `cache_read_tokens`/`cache_write_tokens` fields alongside the
+regular totals, no extra config needed. Self-host's cost estimate prices these separately from a
+regular input token when you've set optional cache rates on that model. Install the matching
+extra:
 
 | Framework             | Install                                      | Integration              |
 | --------------------- | -------------------------------------------- | ------------------------ |
@@ -253,66 +244,55 @@ See **[TRACING.md](TRACING.md)** for the complete Monitor guide.
 
 ---
 
-## Custom agent evaluations
-
-Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine** and **Jaccard** similarity metrics alongside LLM-graded ratings.
-
-```python
-report = (
-    client.evaluations
-    .run(dataset_id="evds_…", subject={"kind": "custom_agent", "framework": "raw_python"})
-    .execute(my_agent_fn)
-    .finalize()
-    .analyze()
-)
+## Self-host
 
-print(report.average_rating)       # LLM-graded score, 0–10
-print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
-print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
+Prefer to run Trace/Evaluate/Monitor locally instead of the hosted dashboard — no account, bring your own LLM keys? This SDK ships a launcher for [AgentX-trace-eval](https://github.com/AgentX-ai/AgentX-trace-eval), a separate, portable governance engine:
 
-print(report.summary)              # AI-generated narrative from .analyze()
-print(report.recommendations)      # list of prioritized, actionable fixes
+```bash
+agentx-trace-eval --dev
 ```
 
-`.analyze()` also generates a full qualitative report (strengths, weaknesses, instruction adherence, reasoning quality, and recommendations), running the same durable, multi-judge pipeline as the dashboard's "Analyze" button. `analyze(mode=..., quality_mode=..., judges=[...])` controls how items are scored and by which models. See [AI analysis report](EVALUATIONS.md#ai-analysis-report) in the full guide for the complete field and parameter reference.
+The first run downloads the engine (and dashboard) into `~/.agentx/bin` and prints a local API key; every run after that just starts it. Point this SDK at it instead of the hosted API:
 
-Ask a case's question several extra ways each run, LLM-paraphrased server-side, to catch agents that break on phrasing rather than substance, and override the judge's prompt/model per config, see [Smoke testing](EVALUATIONS.md#smoke-testing-phrasing-robustness) and [Configuring the judge](EVALUATIONS.md#configuring-the-judge) in the full guide.
+```bash
+export AGENTX_API_BASE_URL=http://localhost:4700/api/v1
+export AGENTX_API_KEY=<printed by agentx-trace-eval on first run>
+```
 
-Since AgentX doesn't own your agent's code, `client.evaluations.prompts` lets AgentX become your prompt's *source of truth* instead — the same problem LangSmith's Prompt Hub and Langfuse's Prompt Management solve. Pull a version at runtime, tag your eval runs (or live traces) with it, and let a judge propose a rewrite from your real worst-rated results — a human always has to approve before it publishes:
+`agentx-trace-eval` isn't this SDK's own code — the engine itself is a separate, compiled binary, downloaded on demand rather than bundled into this package, so installing `agentx-python` doesn't get any heavier for the (much more common) case of just talking to the hosted AgentX API. See that repo's README for what's included, and `AGENTX_INSTALL_DIR`/`AGENTX_TRACE_EVAL_VERSION`/`AGENTX_TRACE_EVAL_SKIP_WEB` env vars to control where/what it installs.
 
-```python
-prompt = client.evaluations.prompts.get("support-agent-system-prompt")  # or prompt.id
-# use prompt.text as your own agent's system prompt
+---
 
-client.evaluations.run(
-    dataset_id="evds_…",
-    subject={"kind": "custom_agent", "metadata": {"promptName": prompt.name}},
-).execute(my_agent_fn)
-```
+## Agents & conversations
 
-See [Prompt registry](EVALUATIONS.md#prompt-registry) in the full guide, or [self-host's docs](https://docs.agentx.so/self-host#prompt-registry) for the "Suggest improvement" dashboard flow (self-host only — no hosted-SaaS equivalent yet).
+Beyond evaluation, tracing, and monitoring, this SDK is also a client for AgentX's own hosted agents — build, chat with, and orchestrate them directly.
 
-See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, smoke testing, judge configuration, prompt registry, and the complete API reference.
+```python
+agent = client.list_agents()[0]
+conversation = agent.new_conversation()
 
----
+# Blocking — returns the full response once it's ready
+print(conversation.chat("What can you help me with?"))
 
-## Self-host
+# Streaming — yields ChatResponse objects as the model produces them
+for chunk in conversation.chat_stream("Hello!"):
+    if chunk.text:
+        print(chunk.text, end="")
+```
 
-Prefer to run Trace/Evaluate/Monitor locally instead of the hosted dashboard — no account, bring your own LLM keys? This SDK ships a launcher for [AgentX-trace-eval](https://github.com/AgentX-ai/AgentX-trace-eval), a separate, portable governance engine:
+Each `ChatResponse` chunk exposes the agent's `text` and, where applicable, its `cot` (chain-of-thought) reasoning, along with any retrieved references and tasks. `agent.list_conversations()` / `conversation.list_messages()` resume history instead of starting fresh.
 
-```bash
-agentx-trace-eval --dev
-```
+A **workforce** is a team of agents coordinated by a designated manager, mixing LLM vendors and routing work between specialists:
 
-The first run downloads the engine (and dashboard) into `~/.agentx/bin` and prints a local API key; every run after that just starts it. Point this SDK at it instead of the hosted API:
+```python
+workforce = client.list_workforces()[0]
+conversation = workforce.new_conversation()
 
-```bash
-export AGENTX_API_BASE_URL=http://localhost:4700/api/v1
-export AGENTX_API_KEY=<printed by agentx-trace-eval on first run>
+for chunk in workforce.chat_stream(conversation.id, "How can you help me with this project?"):
+    if chunk.text:
+        print(chunk.text, end="")
 ```
 
-`agentx-trace-eval` isn't this SDK's own code — the engine itself is a separate, compiled binary, downloaded on demand rather than bundled into this package, so installing `agentx-python` doesn't get any heavier for the (much more common) case of just talking to the hosted AgentX API. See that repo's README for what's included, and `AGENTX_INSTALL_DIR`/`AGENTX_TRACE_EVAL_VERSION`/`AGENTX_TRACE_EVAL_SKIP_WEB` env vars to control where/what it installs.
-
 ---
 
 ## Links

{agentx_python-0.6.13 → agentx_python-0.6.15}/agentx/integrations/_traced_call.py

@@ -77,6 +77,8 @@ def finish_llm_call(
     input_tokens: Optional[int],
     output_tokens: Optional[int],
     error: Optional[str],
+    cache_read_tokens: Optional[int] = None,
+    cache_write_tokens: Optional[int] = None,
 ) -> None:
     """
     Close out one raw-client LLM call — shared by the ``on_finish``/exit
@@ -107,6 +109,8 @@ def finish_llm_call(
             model=model,
             input_tokens=input_tokens,
             output_tokens=output_tokens,
+            cache_read_tokens=cache_read_tokens,
+            cache_write_tokens=cache_write_tokens,
         )
         return
 
@@ -121,4 +125,8 @@ def finish_llm_call(
         span._input_tokens = input_tokens
     if output_tokens:
         span._output_tokens = output_tokens
+    if cache_read_tokens:
+        span._cache_read_tokens = cache_read_tokens
+    if cache_write_tokens:
+        span._cache_write_tokens = cache_write_tokens
     span.__exit__(None, None, None)