synth-agent-sdk 2.2.0__tar.gz → 2.2.2__tar.gz

{synth_agent_sdk-2.2.0 → synth_agent_sdk-2.2.2}/.gitignore

@@ -38,6 +38,13 @@ coverage.xml
 .bedrock_agentcore/
 *.log
 
+# UI server local state (runtime-generated, never commit)
+synth/cli/_ui_assets/conversations.json
+synth/cli/_ui_assets/conversation_agents.json
+synth/cli/_ui_assets/prompts.json
+synth/cli/_ui_assets/evals.json
+synth/cli/_ui_assets/scenarios.json
+
 # CDK build artifacts
 synth-ui-deploy/cdk.out/
 synth-ui-deploy/cdk.out.*/

{synth_agent_sdk-2.2.0 → synth_agent_sdk-2.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: synth-agent-sdk
-Version: 2.2.0
+Version: 2.2.2
 Summary: Autonomous agents, engineered. A Python SDK for building production-grade AI agents and multi-agent systems.
 License-Expression: MIT
 Requires-Python: >=3.10
@@ -62,6 +62,16 @@ Requires-Dist: pytest-asyncio>=0.23; extra == 'cloud-test'
 Requires-Dist: respx>=0.21; extra == 'cloud-test'
 Provides-Extra: datadog
 Requires-Dist: ddtrace>=2.0; extra == 'datadog'
+Provides-Extra: dev
+Requires-Dist: boto3>=1.42.87; extra == 'dev'
+Requires-Dist: hypothesis>=6.0; extra == 'dev'
+Requires-Dist: jsonschema>=4; extra == 'dev'
+Requires-Dist: moto>=5.0; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: pyyaml>=6.0; extra == 'dev'
+Requires-Dist: respx>=0.21; extra == 'dev'
+Requires-Dist: tomli>=2.0; (python_version < '3.11') and extra == 'dev'
 Provides-Extra: enterprise
 Requires-Dist: boto3>=1.42.87; extra == 'enterprise'
 Requires-Dist: opentelemetry-api>=1.20; extra == 'enterprise'
@@ -110,7 +120,7 @@ Description-Content-Type: text/markdown
 
 > Autonomous agents, engineered.
 
-**Version:** 2.2.0 | [PyPI](https://pypi.org/project/synth-agent-sdk/) | [Changelog](CHANGELOG.md)
+**Version:** 2.2.1 | [PyPI](https://pypi.org/project/synth-agent-sdk/) | [Changelog](CHANGELOG.md)
 
 Synth is a Python SDK and platform for building, deploying, and governing production-grade AI agents. It covers the full agent lifecycle — from a 3-line prototype to multi-agent systems running across AWS accounts with policy-as-code guardrails, drift detection, and centralized observability.
 

{synth_agent_sdk-2.2.0 → synth_agent_sdk-2.2.2}/README.md

@@ -2,7 +2,7 @@
 
 > Autonomous agents, engineered.
 
-**Version:** 2.2.0 | [PyPI](https://pypi.org/project/synth-agent-sdk/) | [Changelog](CHANGELOG.md)
+**Version:** 2.2.1 | [PyPI](https://pypi.org/project/synth-agent-sdk/) | [Changelog](CHANGELOG.md)
 
 Synth is a Python SDK and platform for building, deploying, and governing production-grade AI agents. It covers the full agent lifecycle — from a 3-line prototype to multi-agent systems running across AWS accounts with policy-as-code guardrails, drift detection, and centralized observability.
 

synth_agent_sdk-2.2.2/agent/agentcore.yaml

@@ -0,0 +1,29 @@
+# AgentCore Deployment Configuration
+#
+# This file is used by: synth deploy --target agentcore
+#
+# Agent Metadata
+agent_name: agent
+agent_description: "AI agent deployed to AWS AgentCore"
+
+# AWS Configuration
+aws_region: us-east-1
+model_id: 
+cris_enabled: false
+
+# IAM Permissions (least-privilege)
+permissions:
+  - "bedrock:InvokeModel"
+  - "bedrock:InvokeModelWithResponseStream"
+  - "bedrock-agentcore:*"
+  - "ssm:GetParameter"
+  - "secretsmanager:GetSecretValue"
+
+# Runtime Configuration
+runtime:
+  memory_mb: 512
+  timeout_seconds: 300
+
+# Environment Variables (non-sensitive only)
+environment:
+  SYNTH_NO_BANNER: "1"

synth_agent_sdk-2.2.2/agent/handler.py

@@ -0,0 +1,6 @@
+"""AgentCore handler wrapper."""
+
+from agent import agent
+from synth.deploy.agentcore import agentcore_handler
+
+app = agentcore_handler(agent)

synth_agent_sdk-2.2.2/agent_eval/eval_config.json

@@ -0,0 +1,21 @@
+{
+  "config_name": "agent_eval-online-eval",
+  "sampling_rate": 1.0,
+  "evaluators": [
+    {
+      "id": "Builtin.Helpfulness",
+      "arn": "arn:aws:bedrock-agentcore:::evaluator/Builtin.Helpfulness",
+      "level": "TRACE"
+    },
+    {
+      "id": "Builtin.Correctness",
+      "arn": "arn:aws:bedrock-agentcore:::evaluator/Builtin.Correctness",
+      "level": "TRACE"
+    },
+    {
+      "id": "Builtin.GoalSuccessRate",
+      "arn": "arn:aws:bedrock-agentcore:::evaluator/Builtin.GoalSuccessRate",
+      "level": "SESSION"
+    }
+  ]
+}

synth_agent_sdk-2.2.2/agent_eval/eval_dataset.json

@@ -0,0 +1,10 @@
+[
+    {
+        "input": "What is 2 + 2?",
+        "expected": "4"
+    },
+    {
+        "input": "Say hello",
+        "expected": "Hello"
+    }
+]

synth_agent_sdk-2.2.2/community-feedback-analysis.md

@@ -0,0 +1,229 @@
+# Synth SDK — Community Feedback Analysis
+
+> Verified against source as of v2.2.1. Each item is marked **accurate**, **inaccurate**,
+> or **partially accurate** based on a direct code scan. Corrected descriptions replace
+> the original where needed.
+
+---
+
+## Single-Agent Gaps
+
+### 1. Memory is too shallow — ACCURATE (with nuance)
+
+`Memory.thread()`, `Memory.persistent()`, `Memory.semantic()`, and `Memory.dynamodb()`
+cover short-term and session-scoped retrieval. `SemanticMemory` does vector-based
+similarity retrieval, but it is scoped to a single `thread_id`. When backed by an
+external `BaseVectorStore`, embeddings persist across process restarts — but there is
+no built-in mechanism to search *across* thread IDs to surface what a user said in a
+different conversation two weeks ago. `Knowledge` covers RAG over documents; it does
+not cover cross-session episodic memory. This is a real gap for production deployments
+that need long-horizon user context.
+
+---
+
+### 2. Guards are agent-level only — ACCURATE
+
+`ToolFilterGuard` (`Guard.no_tool_calls(patterns)`) blocks tool calls by glob pattern
+at the agent level, but there is no per-tool guard attachment. You cannot say "allow
+`web_search` freely but require approval before `shell`" without splitting into separate
+agents. Per-tool guard policies — where each tool carries its own guard list evaluated
+at invocation time — are not implemented.
+
+---
+
+### 3. Self-correction is implicit, not structured — PARTIALLY ACCURATE
+
+`max_retries` and `retry_backoff` handle transient provider errors. There is no
+structured self-reflection loop where the agent evaluates its own output against a
+rubric and revises before returning.
+
+However, the feedback misses `RalphLoop` — a first-class public primitive
+(`from synth import RalphLoop`) that wraps an agent in a run-test-fix iteration cycle.
+`RalphLoop` re-invokes the agent with a fresh thread on each iteration until an external
+predicate (a shell command or callable) passes or a `max_iterations` budget is exhausted.
+It supports checkpointing, streaming (`astream()`), and crash recovery (`resume()`).
+
+The gap is specifically *self-reflective critique* — where the agent inspects its own
+output against a quality rubric and decides to revise. `RalphLoop` requires an external
+predicate; it does not support the agent reasoning about its own output quality
+internally. That distinction is real and worth calling out.
+
+---
+
+### 4. Tool selection at scale is unsolved — ACCURATE
+
+With 50+ pre-built integrations via `synth-tools`, there is no `ToolSelector`, dynamic
+tool relevance scoring, or per-invocation tool pruning based on task context. Passing
+all tools to the model on every call degrades performance and increases cost. This is a
+real gap.
+
+---
+
+### 5. No adaptive retry strategy — INACCURATE
+
+The feedback claims `retry_backoff` is "a static float." This is wrong. `RetryHandler`
+in `synth/providers/retry.py` implements:
+
+- Exponential backoff: `base * 2^attempt + jitter` (full jitter, not fixed)
+- `Retry-After` header parsing (both integer seconds and HTTP-date format)
+- A `max_duration_seconds` cap that aborts retries before the total wall-clock time
+  exceeds the limit — preventing indefinite blocking on sustained failures
+- Configurable `retryable_status_codes` to extend the default set (408, 429, 5xx)
+
+What is not implemented is *strategy-changing* retries — falling back to a simpler
+tool, decomposing the task differently, or escalating to a human when retries are
+exhausted. The provider fallback chain (`fallback=["gpt-4o", ...]`) handles model-level
+fallback, but task-level strategy changes on retry are not a built-in primitive.
+
+---
+
+### 6. Multimodal at the agent loop level is unclear — ACCURATE
+
+`ImageInput`, `AudioInput`, and `FileInput` exist as dataclasses for annotating tool
+function parameters. They are tool-input types, not native agent-loop multimodal
+inputs. There is no mechanism for a user to send an image mid-conversation and have
+the agent's core reasoning loop process it natively (e.g., as a vision input to the
+provider). The distinction between tool-parameter media and native agent-level
+multimodal reasoning is real and the latter is not implemented.
+
+---
+
+## Multi-Agent Gaps
+
+### 7. No cross-agent shared memory — ACCURATE
+
+Agents in Pipelines, Graphs, and AgentTeams each operate with their own memory scope.
+There is no `SharedMemory` or `TeamContext` primitive that lets agents read each other's
+working state without routing everything through the orchestrator. In a research team
+topology, agents cannot directly access each other's notes. This is a real gap.
+
+---
+
+### 8. Debate and consensus topologies lack conflict resolution mechanics — INACCURATE
+
+The conflict resolution mechanics are fully implemented and configurable:
+
+**Consensus** (`ConsensusExecutor`) supports three strategies via
+`plan.parameters["consensus"]`:
+- `"majority_vote"` — tallies by exact-text equality; ties broken by declaration order;
+  automatically downgrades to `"synthesise"` when no answer repeats (R2.10.4)
+- `"best_of"` — selects the longest (most elaborated) answer
+- `"synthesise"` — calls the orchestrator model with the full set of candidate answers
+  to produce a synthesised resolution
+
+**Debate** (`DebateExecutor`) runs debaters in alternating rounds for up to
+`plan.parameters["max_rounds"]` rounds (default 2), then invokes a configurable judge
+(`plan.parameters["judge"]`). When no judge is configured, the orchestrator model acts
+as the default judge (R2.9.4). The full debate transcript is passed to the judge.
+
+These mechanics are first-class configurable parameters, not hidden inside an opaque
+orchestrator.
+
+---
+
+### 9. No circuit breaker / agent health monitoring — ACCURATE
+
+`DurableRunner` provides step-level journaling and replay after a crash, and
+`max_iterations` prevents infinite graph loops. However, there is no live circuit
+breaker pattern during execution. If a parallel graph branch enters a tight failure
+loop, hits its rate limit repeatedly, or throws on every invocation, there is no
+mechanism to isolate and recover that branch without aborting the whole graph run.
+Live fault isolation — detecting a degraded branch and routing around it while other
+branches continue — is not implemented.
+
+---
+
+### 10. Dynamic team composition isn't supported — INACCURATE
+
+`AgentTeam` supports runtime team modification via `add_agent(agent)` and
+`remove_agent(name)`. Both methods update the internal agent map and rebuild the skill
+registry so subsequent routing decisions see the updated roster. The feedback's claim
+that "team membership is static at definition time" is incorrect.
+
+What is not implemented is *autonomous* dynamic composition — where the orchestrator
+itself decides to instantiate a new specialist agent type that was not defined anywhere
+in the original team configuration. `add_agent` requires the caller to supply a
+pre-built agent instance.
+
+---
+
+### 11. Cost attribution in multi-agent is implicit — PARTIALLY ACCURATE
+
+`CostAttribution` (`synth/tenancy/cost_attribution.py`) provides per-tenant cost
+recording with both in-memory and DynamoDB backends. `RunResult.cost` uses the live
+pricing feed for accuracy.
+
+The gap is per-agent cost budgets *within a single multi-agent run*. `TenantContext`
+scopes cost across runs by tenant. `TeamBudgetExceededError` exists as an error type,
+suggesting budget enforcement is at least partially wired. However, there is no
+first-class API for setting per-agent cost ceilings within a single team execution so
+that one runaway agent cannot consume the entire run budget. This is a real gap for
+production multi-agent deployments.
+
+---
+
+### 12. Streaming across handoffs is unclear — INACCURATE
+
+Pipeline streaming is implemented. `PipelineExecutor.astream()` is a native streaming
+override that opens a per-stage stream iterator, forwards tokens as they arrive, and
+emits `agent_start` / `agent_complete` frames around each stage. Tokens flow through
+to the caller while each stage is still processing — stages do not buffer to completion
+before the next begins. `StageEvent(stage_name, event)` wraps each event so consumers
+know which pipeline stage produced it.
+
+---
+
+### 13. The router only routes by complexity — INACCURATE
+
+There is no `SmartRouter` in the codebase. The feedback's description of
+"complexity-based model tier selection" does not correspond to any implemented class.
+
+The actual routing system is a four-strategy hierarchy:
+- `HybridRouter` (default for `AgentTeam(strategy="auto")`) — composes `RuleRouter`
+  and `LLMRouter`; rules fire first, LLM is the fallback
+- `RuleRouter` — deterministic callable-rule table; user-defined rules with no LLM call
+- `LLMRouter` — prompts the orchestrator model for a structured routing decision
+- `HeuristicRouter` — pure keyword/message-length/agent-role heuristics; no LLM call
+
+Routing dimensions include: task keywords, message length, agent role shape, policy
+pins, allowed/forbidden topology lists, and full LLM reasoning over the task and team
+descriptor. What is not implemented is routing by agent availability/load, agent
+historical track record on similar tasks, or cost-target-based routing. Those are real
+gaps in the routing policy language.
+
+---
+
+## RalphLoop — What It Is
+
+Since the feedback doesn't cover it and you want to talk about it: `RalphLoop` is a
+first-class public primitive for autonomous run-test-fix iteration.
+
+```python
+from synth import Agent, RalphLoop
+
+agent = Agent(model="claude-sonnet-4-5", instructions="Write and fix code.")
+loop = RalphLoop(agent=agent, predicate="pytest -x", max_iterations=10)
+result = loop.run("Implement the features described in PRD.md")
+
+print(result.completed)          # True if tests pass within budget
+print(result.iterations)         # How many iterations it took
+print(result.predicate_failures) # What failed on each iteration
+```
+
+Key properties:
+- **Fresh context per iteration** — each iteration gets a new `thread_id` so the agent
+  cannot get stuck hallucinating fixes to earlier claims
+- **Predicate types** — shell command (exit 0 = done), sync callable, or async callable
+- **Checkpointing** — optional `BaseCheckpointStore` persists state after each iteration;
+  `RalphLoop.resume(run_id, ...)` continues from the last checkpoint after a crash
+- **Streaming** — `loop.astream(prompt)` yields `NodeStartEvent` / `NodeCompleteEvent`
+  per iteration plus a terminal `ralph_summary` event
+- **Hook integration** — `pre_step` and `post_step` hooks fire around each iteration;
+  a `block` decision from `pre_step` aborts the loop cleanly
+- **Hard budget** — `max_iterations` is enforced; the loop returns
+  `RalphResult(completed=False, ...)` rather than raising when the budget runs out
+
+The gap relative to the self-correction feedback (item 3) is that `RalphLoop` requires
+an *external* predicate — the agent does not introspect its own output quality. A
+self-reflective variant where the predicate is itself an LLM call evaluating the output
+against a rubric would be a natural extension.

synth_agent_sdk-2.2.2/dummy.json

@@ -0,0 +1 @@
+[]

synth_agent_sdk-2.2.2/my_agent_eval/eval_config.json

@@ -0,0 +1,21 @@
+{
+  "config_name": "my_agent_eval-online-eval",
+  "sampling_rate": 1.0,
+  "evaluators": [
+    {
+      "id": "Builtin.Helpfulness",
+      "arn": "arn:aws:bedrock-agentcore:::evaluator/Builtin.Helpfulness",
+      "level": "TRACE"
+    },
+    {
+      "id": "Builtin.Correctness",
+      "arn": "arn:aws:bedrock-agentcore:::evaluator/Builtin.Correctness",
+      "level": "TRACE"
+    },
+    {
+      "id": "Builtin.GoalSuccessRate",
+      "arn": "arn:aws:bedrock-agentcore:::evaluator/Builtin.GoalSuccessRate",
+      "level": "SESSION"
+    }
+  ]
+}

synth_agent_sdk-2.2.2/my_agent_eval/eval_dataset.json

@@ -0,0 +1,10 @@
+[
+    {
+        "input": "What is 2 + 2?",
+        "expected": "4"
+    },
+    {
+        "input": "Say hello",
+        "expected": "Hello"
+    }
+]

{synth_agent_sdk-2.2.0 → synth_agent_sdk-2.2.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "synth-agent-sdk"
-version = "2.2.0"
+version = "2.2.2"
 description = "Autonomous agents, engineered. A Python SDK for building production-grade AI agents and multi-agent systems."
 readme = "README.md"
 license = "MIT"
@@ -36,6 +36,20 @@ aws = ["synth-agent-sdk[agentcore]"]
 mcp = ["mcp>=1.0"]
 testing = []
 quickstart = ["anthropic>=0.39", "openai>=1.0"]
+# Full test/dev dependency set used by CI to run the unit + property
+# suites. Includes mocking (moto, respx), property testing (hypothesis),
+# and the pytest stack. Not shipped to end users.
+dev = [
+    "pytest>=8.0",
+    "pytest-asyncio>=0.23",
+    "hypothesis>=6.0",
+    "moto>=5.0",
+    "respx>=0.21",
+    "boto3>=1.42.87",
+    "jsonschema>=4",
+    "pyyaml>=6.0",
+    "tomli>=2.0; python_version < '3.11'",
+]
 ui = ["uvicorn>=0.30", "fastapi>=0.115"]
 cdk = ["aws-cdk-lib>=2.100.0", "constructs>=10.0.0"]
 platform = ["boto3>=1.42.87", "pyyaml>=6.0"]
@@ -78,6 +92,16 @@ all = [
 
 [tool.hatch.build.targets.wheel]
 packages = ["synth"]
+exclude = [
+    # Test files live inside the synth package but must never ship
+    "synth/tests/",
+    # Runtime-generated local state files — never ship in the wheel
+    "synth/cli/_ui_assets/conversations.json",
+    "synth/cli/_ui_assets/conversation_agents.json",
+    "synth/cli/_ui_assets/prompts.json",
+    "synth/cli/_ui_assets/evals.json",
+    "synth/cli/_ui_assets/scenarios.json",
+]
 
 # Explicitly declare the bundled pricing data files so the wheel
 # manifest lists them even if they move or are renamed at release cut
@@ -103,12 +127,14 @@ exclude = [
     "Synth Website/",
     "synth-ui-deploy/",
     "synth-tools/",
+    "synth-community/",
     "power/",
     "SynthAgentSDK Power/",
     "tests/",
     "devtools/",
     "scripts/",
     ".kiro/",
+    ".github/",
     ".vscode/",
     ".venv/",
     ".hypothesis/",

{synth_agent_sdk-2.2.0 → synth_agent_sdk-2.2.2}/synth/agent.py

@@ -619,6 +619,7 @@ class Agent:
         memory_context: Literal[
             "none", "long_term", "episodic", "all"
         ] = "none",
+        tool_cap_behaviour: Literal["generate", "force"] = "generate",
     ) -> None:
         # Load unified config (Req 25) — values are defaults that can
         # be overridden by constructor args and env vars.
@@ -637,6 +638,7 @@ class Agent:
         self.deps = deps
         self.parallel_guards = parallel_guards
         self.max_tool_calls = max_tool_calls
+        self._tool_cap_behaviour = tool_cap_behaviour
 
         # Agent lifecycle hooks (Req 1) — copied to defend against
         # mutation of the caller's list after construction.
@@ -2557,6 +2559,49 @@ class Agent:
                 "post_step", messages, tool_round, response.text or "",
             )
 
+        # ── Force synthesis when tool cap fires ──────────────────────
+        # If the loop exited because max_tool_rounds was reached (not
+        # because the model voluntarily stopped), the model still wanted
+        # to call tools.  When tool_cap_behaviour="generate", make one
+        # final call WITHOUT tools available, instructing the model to
+        # synthesise its findings.  This guarantees a non-empty
+        # response.text for the RunResult.
+        if (
+            response.tool_calls
+            and tool_round >= max_tool_rounds
+            and self._tool_cap_behaviour == "generate"
+        ):
+            # Append the assistant's unfulfilled tool request as context
+            messages.append({
+                "role": "assistant",
+                "content": response.text or "",
+            })
+            messages.append({
+                "role": "user",
+                "content": (
+                    "[SYSTEM] Tool call limit reached. You have gathered "
+                    "sufficient information from your previous tool calls. "
+                    "Produce your final answer now based on all the data "
+                    "you have collected. Do not request any more tools."
+                ),
+            })
+
+            # Final call with tools stripped — model MUST produce text.
+            async with collector.span(
+                active_model,
+                "llm_call",
+                tenant_id=(tenant_ctx.tenant_id if tenant_ctx else None),
+            ) as synth_meta:
+                response = await self.retry_handler.execute(
+                    active_provider.complete,
+                    messages,
+                    tools=[],
+                    prompt_caching=self._prompt_caching,
+                )
+                synth_meta["total_tokens"] = response.usage.total_tokens
+                synth_meta["input_tokens"] = response.usage.input_tokens
+                synth_meta["output_tokens"] = response.usage.output_tokens
+
         # 5. Output guard evaluation
         await self._evaluate_guards(
             response.text,
@@ -2902,6 +2947,10 @@ class Agent:
             # Resolve active provider (with fallback on initial failure)
             active_model, active_provider = self.model, self.provider
             first_call = True
+            _stream_tool_round = 0
+            _stream_max_rounds = self.max_tool_calls
+            _stream_hit_cap = False
+            _stream_is_initial = True
 
             while True:
                 needs_another_round = False
@@ -3114,6 +3163,80 @@ class Agent:
                 if not needs_another_round:
                     break
 
+                # Track tool rounds: the first iteration is the initial
+                # provider call; subsequent iterations are tool rounds.
+                # Only count after the initial call has been made.
+                if _stream_is_initial:
+                    _stream_is_initial = False
+                else:
+                    _stream_tool_round += 1
+
+                if _stream_tool_round >= _stream_max_rounds:
+                    _stream_hit_cap = True
+                    break
+
+            # ── Force synthesis when tool cap fires (streaming) ──────
+            # Mirrors the _arun_inner synthesis logic for the streaming
+            # path.  When the stream loop exits at the cap and
+            # tool_cap_behaviour="generate", make one final streaming
+            # call with tools stripped to produce synthesis text.
+            if _stream_hit_cap and self._tool_cap_behaviour == "generate":
+                messages.append({
+                    "role": "assistant",
+                    "content": accumulated_text or "",
+                })
+                messages.append({
+                    "role": "user",
+                    "content": (
+                        "[SYSTEM] Tool call limit reached. You have gathered "
+                        "sufficient information from your previous tool calls. "
+                        "Produce your final answer now based on all the data "
+                        "you have collected. Do not request any more tools."
+                    ),
+                })
+
+                # Reset accumulated text — the synthesis IS the response
+                accumulated_text = ""
+
+                async with collector.span(
+                    active_model, "llm_call",
+                ) as synth_meta:
+                    async for event in active_provider.stream(
+                        messages, tools=[],
+                    ):
+                        if isinstance(event, TextChunkEvent):
+                            accumulated_text += event.text
+                            yield TokenEvent(text=event.text)
+
+                        elif isinstance(event, ProviderDoneEvent):
+                            total_usage = TokenUsage(
+                                input_tokens=(
+                                    total_usage.input_tokens
+                                    + event.usage.input_tokens
+                                ),
+                                output_tokens=(
+                                    total_usage.output_tokens
+                                    + event.usage.output_tokens
+                                ),
+                                total_tokens=(
+                                    total_usage.total_tokens
+                                    + event.usage.total_tokens
+                                ),
+                            )
+                            synth_meta["total_tokens"] = (
+                                event.usage.total_tokens
+                            )
+                            synth_meta["input_tokens"] = (
+                                event.usage.input_tokens
+                            )
+                            synth_meta["output_tokens"] = (
+                                event.usage.output_tokens
+                            )
+
+                        elif isinstance(event, ProviderErrorEvent):
+                            yield ErrorEvent(error=event.error)
+                            return
+
             # Finalise trace and build RunResult, then yield DoneEvent
             trace = collector.finalise()
             elapsed_ms = (time.perf_counter() - start_time) * 1000