agentbyte 0.42.4__tar.gz → 0.44.0__tar.gz

{agentbyte-0.42.4 → agentbyte-0.44.0}/CHANGELOG.md

@@ -6,6 +6,59 @@ The format follows Keep a Changelog principles and semantic versioning.
 
 ## [Unreleased]
 
+## [0.44.0] - 2026-10-05
+
+### Changed
+
+- Fixed string option sets are `StrEnum`s (spec 0067). Plain strings are still accepted
+  everywhere (`AgentHistoryPolicy(window="run")` keeps working) and serialize to the same values.
+  New enums: `HistoryWindow`, `HistoryFormat`, `SelectorWindow`, `SelectorFormat`, `StripMode`,
+  `FallbackStrategy`, `TurnKind` (`agentbyte.orchestration`); `OrchestratorPattern`
+  (`agentbyte.presets`); `CompositeMode` (`agentbyte.termination`); `PairwiseWinner`
+  (`agentbyte.eval`); `TaskType` (`agentbyte.eval.history`); `CheckpointType`, `WorkflowMemoryMode`
+  (`agentbyte.workflow`); `GraphView`, `EntitySource`, `EntityType`, `SafeSearch`, `PageFormat`,
+  `RetrySequenceOutcome`, `RetryAttemptOutcome`. The literal aliases `ScoringStatus`, `RunStatus`,
+  `ItemState`, `EvidenceStatus`, `ItemOutcome`, `ExecutionStatus`, `PricingStatus`, `GroupKind`
+  are now `StrEnum` classes with the same names (iterate them instead of `typing.get_args`).
+- The 16 `class X(str, Enum)` classes (workflow statuses and modes, `OperationType`, optim
+  `Operation`/`ProposalMode`, eval splitting) are `StrEnum`: `str(member)` and f-strings print the
+  value (`"completed"`) instead of `"StepStatus.COMPLETED"`.
+- Unknown values for `HandoffOrchestrator(fallback_strategy=...)`, `CompositeTermination(mode=...)`,
+  `WorkflowAgent(memory_mode=...)`, `WorkflowVisualizer` `view` and the research tools'
+  `default_safe_search` raise `ValueError` instead of silently using a default branch.
+- `WebFetchTool` / `ExtractTextTool` return a failed `ToolResult` listing the valid values for an
+  unknown model-supplied `output_format` (previously silently plain text).
+- `SuiteItemVerdict.state/scoring_status/execution_status/evidence_status`,
+  `EvalItemRecord.execution_status` and `WorkflowCheckpoint.checkpoint_type` are typed with their
+  enums instead of bare `str`.
+
+### Fixed
+
+- OTel span-name fallback rendered `"OperationType.MEMORY_ACCESS <agent>"`, and the logger printed
+  workflow event types as `"WorkflowEventType.…"`; both print the value now.
+- `copy_tool_metadata` accepts `StrEnum` members (stored as their plain value).
+
+## [0.43.0] - 2026-10-05
+
+### Added
+
+- Agents: `stop_at_tools=[...]` ends the run with `finish_reason="tool_executed"`, without a
+  final model call, once a call to a named tool succeeds. Multi-step tool discovery and retries
+  after a failed call still run; a failed stop tool does not stop the run; the budget-exhaustion
+  finalization still applies. Names are validated against the agent's tools at construction.
+  Meant for team specialists whose raw tool results a synthesizer turns into the answer.
+- Orchestration: `AgentHistoryPolicy(window="run")` sends this run's messages in full and
+  condenses earlier session turns to each user message plus the answer that closed it (no prior
+  tool calls, tool results, or worker notes). Use it as `FinalResultPolicy.synthesis_history` so a
+  synthesizer's input stays bounded across follow-up turns without losing their context.
+- Orchestration: workers that end with `max_iterations` / `max_iterations_finalized` are reported to
+  the synthesizer as a separate completion fact (partial findings), next to error turns.
+
+### Fixed
+
+- Design guide: `summarize_tool_result` was documented as a summarization knob; it actually ends
+  the run after the first tool batch. The agent topic now describes it and `stop_at_tools`.
+
 ## [0.42.4] - 2026-10-05
 
 ### Fixed

{agentbyte-0.42.4 → agentbyte-0.44.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: agentbyte
-Version: 0.42.4
+Version: 0.44.0
 Summary: A toolkit for designing multiagent systems
 Author-email: MrDataPsycho <mr.data.psycho@gmail.com>
 License-Expression: LicenseRef-Proprietary
@@ -77,7 +77,7 @@ Description-Content-Type: text/markdown
 
 Agentbyte is an observability-first agentic AI framework for building and studying multiagent systems with a learning-first, implementation-oriented workflow.
 
-Current release: **0.42.4**
+Current release: **0.44.0**
 
 ## Building an Agent
 

{agentbyte-0.42.4 → agentbyte-0.44.0}/README.md

@@ -2,7 +2,7 @@
 
 Agentbyte is an observability-first agentic AI framework for building and studying multiagent systems with a learning-first, implementation-oriented workflow.
 
-Current release: **0.42.4**
+Current release: **0.44.0**
 
 ## Building an Agent
 

agentbyte-0.44.0/src/agentbyte/__about__.py

@@ -0,0 +1,2 @@
+__version__ = "0.44.0"
+VERSION = __version__

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/agents/agent.py

@@ -372,6 +372,14 @@ class Agent(BaseAgent):
             and working_context.get_approval_response(tool_call.call_id) is None
         )
 
+    def _is_stop_tool_success(self, item: object) -> bool:
+        """Whether *item* is a successful result of a ``stop_at_tools`` tool."""
+        return (
+            isinstance(item, ToolMessage)
+            and item.success
+            and item.tool_name in self.stop_at_tools
+        )
+
     async def _resolve_tool_batch(
         self,
         working_context: AgentContext,
@@ -560,6 +568,7 @@ class Agent(BaseAgent):
                 # not-yet-run siblings, in order) before asking the model again.
                 if self._unresolved_tail_calls(working_context):
                     outcome: dict[str, int | bool] = {"executed": 0, "paused": False}
+                    stop_tool_succeeded = False
                     async for item in self._resolve_tool_batch(
                         working_context,
                         cancellation_token,
@@ -567,11 +576,15 @@ class Agent(BaseAgent):
                         provider_tools,
                         outcome,
                     ):
+                        stop_tool_succeeded |= self._is_stop_tool_success(item)
                         yield item
                     tool_calls += int(outcome["executed"])
                     if outcome["paused"]:
                         finish_reason = "approval_needed"
                         break
+                    if stop_tool_succeeded:
+                        finish_reason = "tool_executed"
+                        break
 
                 llm_messages = await self._prepare_llm_messages([], working_context, extra_instructions=extra_instructions)
                 if self.memory is not None:
@@ -773,6 +786,7 @@ class Agent(BaseAgent):
                     break
 
                 outcome = {"executed": 0, "paused": False}
+                stop_tool_succeeded = False
                 async for item in self._resolve_tool_batch(
                     working_context,
                     cancellation_token,
@@ -780,13 +794,14 @@ class Agent(BaseAgent):
                     provider_tools,
                     outcome,
                 ):
+                    stop_tool_succeeded |= self._is_stop_tool_success(item)
                     yield item
                 tool_calls += int(outcome["executed"])
                 if outcome["paused"]:
                     finish_reason = "approval_needed"
                     break
 
-                if not self.summarize_tool_result:
+                if stop_tool_succeeded or not self.summarize_tool_result:
                     finish_reason = "tool_executed"
                     break
 

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/agents/base.py

@@ -59,6 +59,7 @@ class BaseAgent(ABC):
         required_tools: Optional[List[str]] = None,
         example_tasks: Optional[List[str]] = None,
         context_providers: Optional[List[ContextProvider]] = None,
+        stop_at_tools: Optional[List[str]] = None,
         **kwargs: Any,
     ) -> None:
         """Initialize the agent.
@@ -69,6 +70,15 @@ class BaseAgent(ABC):
         ``max_iterations >= 2``: with ``max_iterations=1`` the only turn keeps
         its tools, so a run that calls a tool ends with
         ``finish_reason="max_iterations"`` and no final answer.
+
+        ``summarize_tool_result=False`` ends the run right after the first tool
+        batch executes, with no further model call
+        (``finish_reason="tool_executed"``). ``stop_at_tools`` is the selective
+        form: the agent keeps its tool loop (multi-step discovery, retries after a
+        failed call) and ends the same way once a call to one of the named tools
+        succeeds. The names must be the agent's own tools. Use either when another
+        component, such as an orchestrator's synthesizer, writes the answer from
+        the raw tool results.
         """
         self.name = name
         self.description = description
@@ -84,6 +94,7 @@ class BaseAgent(ABC):
         self.required_tools = required_tools or []
         self.example_tasks = example_tasks or []
         self.context_providers: List[ContextProvider] = list(context_providers) if context_providers else []
+        self.stop_at_tools: List[str] = list(stop_at_tools) if stop_at_tools else []
         self.extra_config = kwargs
 
         self._validate_configuration()
@@ -100,6 +111,13 @@ class BaseAgent(ABC):
             raise AgentConfigurationError("Model client is required")
         if self.max_iterations <= 0:
             raise AgentConfigurationError("max_iterations must be > 0")
+        tool_names = {tool.name for tool in self.tools}
+        unknown = [name for name in self.stop_at_tools if name not in tool_names]
+        if unknown:
+            raise AgentConfigurationError(
+                f"stop_at_tools names unknown tools {unknown}; "
+                f"available: {sorted(tool_names)}"
+            )
 
     def _process_tools(self, tools: List[Union[BaseTool, Callable]]) -> List[BaseTool]:
         """Normalize mixed tool list into BaseTool instances."""

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/compaction/grouping.py

@@ -3,7 +3,8 @@
 from __future__ import annotations
 
 from dataclasses import dataclass
-from typing import Any, Literal
+from enum import StrEnum
+from typing import Any
 
 from agentbyte.compaction.errors import PendingToolExchangeError, ToolExchangeError
 from agentbyte.messages import (
@@ -14,7 +15,15 @@ from agentbyte.messages import (
     UserMessage,
 )
 
-GroupKind = Literal["system", "user", "assistant", "tool_exchange", "opaque"]
+
+class GroupKind(StrEnum):
+    """What an indivisible message group contains."""
+
+    SYSTEM = "system"
+    USER = "user"
+    ASSISTANT = "assistant"
+    TOOL_EXCHANGE = "tool_exchange"
+    OPAQUE = "opaque"
 
 
 @dataclass(frozen=True)

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/__init__.py

@@ -58,6 +58,7 @@ from .types import (
     MultiTurnEvalTask,
     WorkflowEvalTask,
     PairwiseResult,
+    PairwiseWinner,
 )
 
 __all__ = [
@@ -76,6 +77,7 @@ __all__ = [
     "BaseEvalRunner",
     "JudgeScoringError",
     "ScoringStatus",
+    "PairwiseWinner",
     "ExactMatchJudge",
     "ContainsJudge",
     "FuzzyMatchJudge",

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/history/__init__.py

@@ -30,6 +30,7 @@ from .models import (
     Redactor,
     RunStatus,
     StructuredPayload,
+    TaskType,
 )
 from .recorder import EvalRunRecorder
 from .sqlite import SQLiteEvalRunStore
@@ -71,5 +72,6 @@ __all__ = [
     "SQLiteEvalRunStore",
     "SqlEvalRunStore",
     "StructuredPayload",
+    "TaskType",
     "history_metadata",
 ]

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/history/conformance.py

@@ -22,7 +22,7 @@ from pydantic import BaseModel
 
 from agentbyte.eval.base import BaseEvalJudge, BaseEvalTarget, JudgeScoringError
 from agentbyte.eval.runner import EvalRunner
-from agentbyte.eval.types import EvalScore, EvalTask, EvalTrajectory, WorkflowEvalTask
+from agentbyte.eval.types import EvalScore, EvalTask, EvalTrajectory, ScoringStatus, WorkflowEvalTask
 from agentbyte.execution_trace.models import ExecutionEvent, ExecutionSpan, ExecutionTrace
 from agentbyte.messages import AssistantMessage, UserMessage, Usage
 
@@ -152,7 +152,7 @@ async def check_lifecycle_updates_counters_and_revisions(store: EvalRunStore) ->
     assert (saved.state, saved.revision, saved.overall) == ("finished", 4, 6.0)
     assert saved.started_at == running.started_at and saved.finished_at is not None
     failed = await store.save_item(_outcome(run, 1).model_copy(update={
-        "scoring_status": "failed", "overall": None, "failure_reason": "target_error",
+        "scoring_status": ScoringStatus.FAILED, "overall": None, "failure_reason": "target_error",
     }))
     assert failed.revision == 5
 

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/history/models.py

@@ -13,12 +13,13 @@ import re
 from collections.abc import Callable
 from datetime import date, datetime, timezone
 from enum import StrEnum
-from typing import Any, Literal
+from typing import Any
 from uuid import uuid4
 
 from pydantic import BaseModel, ConfigDict, Field
 
 from agentbyte.eval.types import EvalScore, EvalTask, MultiTurnEvalTask, ScoringStatus, WorkflowEvalTask
+from agentbyte.execution_trace.models import ExecutionStatus
 from agentbyte.messages import Usage
 
 from .errors import EvalSchemaVersionError, EvalSerializationError
@@ -29,8 +30,26 @@ SCHEMA_VERSION = 1
 EVIDENCE_VERSION = 1
 """Version of the serialized full-capture evidence envelope."""
 
-RunStatus = Literal["queued", "running", "completed", "failed", "cancelled", "interrupted"]
-ItemState = Literal["pending", "running", "finished", "cancelled", "not_started"]
+
+class RunStatus(StrEnum):
+    """Lifecycle status of a recorded evaluation run."""
+
+    QUEUED = "queued"
+    RUNNING = "running"
+    COMPLETED = "completed"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+    INTERRUPTED = "interrupted"
+
+
+class ItemState(StrEnum):
+    """Lifecycle state of one recorded evaluation item."""
+
+    PENDING = "pending"
+    RUNNING = "running"
+    FINISHED = "finished"
+    CANCELLED = "cancelled"
+    NOT_STARTED = "not_started"
 
 
 class CaptureMode(StrEnum):
@@ -45,11 +64,29 @@ class CaptureMode(StrEnum):
     FULL = "full"
 
 
-EvidenceStatus = Literal["not_captured", "captured", "redacted", "unavailable"]
-TaskType = Literal["task", "workflow", "multi_turn"]
+class EvidenceStatus(StrEnum):
+    """Whether an item's full evidence is stored."""
+
+    NOT_CAPTURED = "not_captured"
+    CAPTURED = "captured"
+    REDACTED = "redacted"
+    UNAVAILABLE = "unavailable"
+
+
+class TaskType(StrEnum):
+    """Kind of evaluation task an item ran."""
+
+    TASK = "task"
+    WORKFLOW = "workflow"
+    MULTI_TURN = "multi_turn"
+
 
-ACTIVE_RUN_STATUSES: tuple[RunStatus, ...] = ("queued", "running")
-TERMINAL_ITEM_STATES: tuple[ItemState, ...] = ("finished", "cancelled", "not_started")
+ACTIVE_RUN_STATUSES: tuple[RunStatus, ...] = (RunStatus.QUEUED, RunStatus.RUNNING)
+TERMINAL_ITEM_STATES: tuple[ItemState, ...] = (
+    ItemState.FINISHED,
+    ItemState.CANCELLED,
+    ItemState.NOT_STARTED,
+)
 
 Redactor = Callable[[EvalScore], EvalScore]
 
@@ -83,7 +120,7 @@ class EvalRunRecord(BaseModel):
 
     id: str = Field(default_factory=lambda: str(uuid4()))
     schema_version: int = SCHEMA_VERSION
-    status: RunStatus = "queued"
+    status: RunStatus = RunStatus.QUEUED
     revision: int = 0
     capture: CaptureMode = CaptureMode.SUMMARY
     dataset_id: str
@@ -134,8 +171,8 @@ class EvalItemRecord(BaseModel):
     run_id: str
     item_index: int
     task_name: str
-    task_type: TaskType = "task"
-    state: ItemState = "pending"
+    task_type: TaskType = TaskType.TASK
+    state: ItemState = ItemState.PENDING
     revision: int = 0
     started_at: datetime | None = None
     finished_at: datetime | None = None
@@ -145,10 +182,10 @@ class EvalItemRecord(BaseModel):
     failure_reason: str | None = None
     error_type: str | None = None
     success: bool | None = None
-    execution_status: str | None = None
+    execution_status: ExecutionStatus | None = None
     usage: Usage | None = None
     trace_id: str | None = None
-    evidence_status: EvidenceStatus = "not_captured"
+    evidence_status: EvidenceStatus = EvidenceStatus.NOT_CAPTURED
     evidence: EvalScore | None = None
 
     @property

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/history/recorder.py

@@ -318,9 +318,9 @@ class EvalRunRecorder:
 
     def _outcome(self, run_id: str, index: int, task: EvalTask, score: EvalScore) -> EvalItemRecord:
         evidence: EvalScore | None = None
-        evidence_status: EvidenceStatus = "not_captured"
+        evidence_status: EvidenceStatus = EvidenceStatus.NOT_CAPTURED
         if self.capture == CaptureMode.FULL:
-            evidence, evidence_status = score, "captured"
+            evidence, evidence_status = score, EvidenceStatus.CAPTURED
             if self.redact is not None:
                 try:
                     evidence = self.redact(score)

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/history/transitions.py

@@ -10,9 +10,14 @@ from __future__ import annotations
 from typing import Any
 
 from .errors import EvalHistoryConflictError, EvalQueueFullError
-from .models import EvalItemRecord, EvalRunRecord, RunStatus, utc_now
+from .models import EvalItemRecord, EvalRunRecord, ItemState, RunStatus, utc_now
 
-FINAL_RUN_STATUSES: tuple[RunStatus, ...] = ("completed", "failed", "cancelled", "interrupted")
+FINAL_RUN_STATUSES: tuple[RunStatus, ...] = (
+    RunStatus.COMPLETED,
+    RunStatus.FAILED,
+    RunStatus.CANCELLED,
+    RunStatus.INTERRUPTED,
+)
 
 def admitted_duplicate(existing: EvalRunRecord, run: EvalRunRecord, planned: int) -> EvalRunRecord:
     same = (
@@ -44,7 +49,7 @@ def admit(run: EvalRunRecord, items: list[EvalItemRecord]) -> tuple[EvalRunRecor
         raise ValueError("items must be indexed 0..n-1 in order")
     if any(item.run_id != run.id or item.state != "pending" for item in items):
         raise ValueError("items must be pending and belong to the run")
-    admitted = run.model_copy(update={"status": "queued", "revision": 1, "planned_count": len(items)})
+    admitted = run.model_copy(update={"status": RunStatus.QUEUED, "revision": 1, "planned_count": len(items)})
     return admitted, [item.model_copy(update={"revision": 1}) for item in items]
 
 
@@ -52,7 +57,7 @@ def start(run: EvalRunRecord, worker_id: str | None) -> EvalRunRecord:
     if run.status != "queued":
         raise EvalHistoryConflictError(f"Run {run.id} is {run.status}, not queued")
     return run.model_copy(update={
-        "status": "running", "revision": run.revision + 1,
+        "status": RunStatus.RUNNING, "revision": run.revision + 1,
         "started_at": utc_now(), "worker_id": worker_id,
     })
 
@@ -65,7 +70,7 @@ def item_running(run: EvalRunRecord, item: EvalItemRecord) -> tuple[EvalRunRecor
     revision = run.revision + 1
     return (
         run.model_copy(update={"revision": revision}),
-        item.model_copy(update={"state": "running", "revision": revision, "started_at": utc_now()}),
+        item.model_copy(update={"state": ItemState.RUNNING, "revision": revision, "started_at": utc_now()}),
     )
 
 
@@ -125,14 +130,14 @@ def finish(
     closed: list[EvalItemRecord] = []
     for item in open_items:
         if item.state == "pending":
-            update: dict[str, Any] = {"state": "not_started"}
+            update: dict[str, Any] = {"state": ItemState.NOT_STARTED}
         else:
-            update = {"state": "cancelled", "failure_reason": "cancelled", "finished_at": now}
+            update = {"state": ItemState.CANCELLED, "failure_reason": "cancelled", "finished_at": now}
         closed.append(item.model_copy(update={**update, "revision": revision}))
     updated = run.model_copy(update={"revision": revision})
     for item in closed:
         updated = updated.model_copy(update=_count(updated, item))
-    return updated.model_copy(update={"status": status, "finished_at": now, "error": error}), closed
+    return updated.model_copy(update={"status": RunStatus(status), "finished_at": now, "error": error}), closed
 
 
 def check_same_evidence(item: EvalItemRecord, stored: str | None, new: str | None) -> None:

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/judges/pairwise.py

@@ -21,6 +21,7 @@ from agentbyte.eval.types import (
     EvalScore,
     EvalTrajectory,
     PairwiseResult,
+    PairwiseWinner,
 )
 
 
@@ -129,16 +130,20 @@ class PairwiseJudge(BaseEvalJudge):
             parsed = self._parse_response(result.structured_output, result.message.content)
         except (ValidationError, json.JSONDecodeError, TypeError) as exc:
             raise JudgeScoringError("invalid_response", "Malformed pairwise response") from exc
-        winner = parsed.winner.strip().lower()
-        if winner not in ("a", "b", "tie"):
-            raise JudgeScoringError("invalid_response", f"Unknown winner: {parsed.winner!r}")
+        # The model's raw label stays a str until normalized and validated here.
+        try:
+            winner = PairwiseWinner(parsed.winner.strip().lower())
+        except ValueError:
+            raise JudgeScoringError(
+                "invalid_response", f"Unknown winner: {parsed.winner!r}"
+            ) from None
         margin = float(parsed.margin)
         if not math.isfinite(margin) or not 0.0 <= margin <= 1.0:
             raise JudgeScoringError("invalid_score", f"Margin outside 0-1: {parsed.margin!r}")
         return PairwiseResult(
             task=trajectory_a.task,
-            winner=winner,  # type: ignore[arg-type]
-            margin=0.0 if winner == "tie" else margin,
+            winner=winner,
+            margin=0.0 if winner == PairwiseWinner.TIE else margin,
             reasoning=parsed.reasoning,
             trajectory_a=trajectory_a,
             trajectory_b=trajectory_b,

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/splitting.py

@@ -8,7 +8,7 @@ import math
 from abc import ABC, abstractmethod
 from collections import Counter
 from collections.abc import Sequence
-from enum import Enum
+from enum import StrEnum
 from typing import Any
 
 from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
@@ -16,14 +16,14 @@ from pydantic import BaseModel, ConfigDict, Field, field_validator, model_valida
 from agentbyte.eval.types import EvalTask
 
 
-class EvalSplitStrategy(str, Enum):
+class EvalSplitStrategy(StrEnum):
     """Supported holdout partition layouts."""
 
     TRAIN_TEST = "train_test"
     TRAIN_VALIDATION_TEST = "train_validation_test"
 
 
-class EvalPartition(str, Enum):
+class EvalPartition(StrEnum):
     """Named partitions produced by an eval split."""
 
     TRAIN = "train"

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/targets/model.py

@@ -5,6 +5,7 @@ from __future__ import annotations
 import time
 
 from agentbyte.cancellation_token import CancellationToken
+from agentbyte.execution_trace import ExecutionStatus
 from agentbyte.llm import BaseChatCompletionClient
 from agentbyte.messages import SystemMessage, Usage, UserMessage
 
@@ -36,7 +37,7 @@ class ModelEvalTarget(BaseEvalTarget):
                 task,
                 start_time,
                 error="Execution cancelled before model call",
-                status="cancelled",
+                status=ExecutionStatus.CANCELLED,
             )
 
         messages = []
@@ -70,7 +71,7 @@ class ModelEvalTarget(BaseEvalTarget):
         start_time: float,
         *,
         error: str,
-        status: str = "failed",
+        status: ExecutionStatus = ExecutionStatus.FAILED,
     ) -> EvalTrajectory:
         elapsed_ms = int((time.perf_counter() - start_time) * 1000)
         return EvalTrajectory(

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/targets/runtime.py

@@ -9,7 +9,7 @@ from collections.abc import AsyncIterator
 from typing import Any
 from weakref import WeakKeyDictionary
 
-from agentbyte.execution_trace import TraceCollector
+from agentbyte.execution_trace import ExecutionStatus, TraceCollector
 from agentbyte.messages import Usage
 from agentbyte.eval.types import EvalTask, EvalTrajectory
 
@@ -34,18 +34,19 @@ def build_trajectory(
     started: float,
     *,
     messages: list,
-    status: str,
+    status: ExecutionStatus | str,
     error: str | None = None,
     usage: Usage | None = None,
     **metadata: Any,
 ) -> EvalTrajectory:
+    status = ExecutionStatus(status)
     elapsed = int((time.perf_counter() - started) * 1000)
     total = usage if usage is not None else collector.trace.aggregate_usage()
     total = total.model_copy(update={"duration_ms": elapsed})
     return EvalTrajectory(
         task=task,
         messages=messages,
-        success=status == "completed",
+        success=status == ExecutionStatus.COMPLETED,
         error=error,
         usage=total,
         trace=collector.trace,

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/types.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 
 import math
 from enum import StrEnum
-from typing import Any, Literal
+from typing import Any
 
 from pydantic import (
     BaseModel,
@@ -130,7 +130,20 @@ class EvalTrajectory(BaseModel):
         return value
 
 
-ScoringStatus = Literal["scored", "failed", "cancelled"]
+class ScoringStatus(StrEnum):
+    """Whether a score (or comparison) is valid measured evidence."""
+
+    SCORED = "scored"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class PairwiseWinner(StrEnum):
+    """Which trajectory a pairwise comparison preferred."""
+
+    A = "a"
+    B = "b"
+    TIE = "tie"
 
 
 # Markers written by pre-0055 judges on placeholder scores. Such records are
@@ -158,7 +171,7 @@ class EvalScore(BaseModel):
         description="Overall score on a 0-10 scale; None when not scored",
     )
     scoring_status: ScoringStatus = Field(
-        default="scored",
+        default=ScoringStatus.SCORED,
         description="Whether this outcome is a valid measurement",
     )
     failure_reason: str | None = Field(
@@ -251,14 +264,14 @@ class PairwiseResult(BaseModel):
 
     task: EvalTask = Field(..., description="Task both trajectories were evaluated on")
     comparison_status: ScoringStatus = Field(
-        default="scored",
+        default=ScoringStatus.SCORED,
         description="Whether this comparison is a valid measured preference",
     )
     failure_reason: str | None = Field(
         default=None,
         description="Safe reason code for failed or cancelled comparisons",
     )
-    winner: Literal["a", "b", "tie"] | None = Field(
+    winner: PairwiseWinner | None = Field(
         default=None,
         description="Which trajectory was better, or tie; None when not compared",
     )

{agentbyte-0.42.4 → agentbyte-0.44.0}/src/agentbyte/eval/workbench/smoke.py

@@ -18,7 +18,7 @@ from agentbyte.cancellation_token import CancellationToken
 from agentbyte.eval.base import BaseEvalTarget
 from agentbyte.eval.judges import ContainsJudge
 from agentbyte.eval.types import EvalTask, EvalTrajectory
-from agentbyte.execution_trace.models import ExecutionSpan, ExecutionTrace
+from agentbyte.execution_trace.models import ExecutionSpan, ExecutionStatus, ExecutionTrace
 from agentbyte.messages import AssistantMessage, UserMessage
 
 from .suite import EvalSuite, EvalSuiteRegistry
@@ -40,11 +40,11 @@ class ArithmeticTarget(BaseEvalTarget):
         span = ExecutionSpan(kind="agent", name=self.name, input=task.input)
         match = _QUESTION.search(task.input)
         if match is None:
-            answer, status, error = None, "failed", "unsupported_question"
+            answer, status, error = None, ExecutionStatus.FAILED, "unsupported_question"
         else:
             left, symbol, right = match.groups()
             answer = str(_OPERATORS[symbol](int(left), int(right)))
-            status, error = "completed", None
+            status, error = ExecutionStatus.COMPLETED, None
         span = span.model_copy(update={"status": status, "output": answer})
         messages = [UserMessage(content=task.input, source="user")]
         if answer is not None: