agent-tco 0.0.1__tar.gz

agent_tco-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Chinmay Kakatkar
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agent_tco-0.0.1/PKG-INFO

@@ -0,0 +1,145 @@
+Metadata-Version: 2.3
+Name: agent-tco
+Version: 0.0.1
+Summary: agent-tco: Pareto-optimal configuration of agentic workflows.
+Author: Chinmay Kakatkar
+License: MIT License
+         
+         Copyright (c) 2026 Chinmay Kakatkar
+         
+         Permission is hereby granted, free of charge, to any person obtaining a copy
+         of this software and associated documentation files (the "Software"), to deal
+         in the Software without restriction, including without limitation the rights
+         to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+         copies of the Software, and to permit persons to whom the Software is
+         furnished to do so, subject to the following conditions:
+         
+         The above copyright notice and this permission notice shall be included in all
+         copies or substantial portions of the Software.
+         
+         THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+         IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+         FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+         AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+         LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+         OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+         SOFTWARE.
+Requires-Dist: scipy>=1.10
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/ckstash/agent-tco
+Description-Content-Type: text/markdown
+
+# agent-tco
+
+**agent-tco** is a Python library for Pareto-optimal configuration of agentic workflows.
+
+---
+
+## Features
+
+- **Workflow simulator** with six-class scenario taxonomy (i1–i6): happy path, retry-resolved, HITL success, HITL partial, compound recovery, aborted.
+- **Grid search** — evaluates all candidate configurations with Beta credible intervals on scenario probabilities.
+- **Pareto frontier** — identifies non-dominated configurations on the ACR vs. E[TCO] plane.
+- **Critical step sensitivity analysis** — Proposition 1: ∂ACR/∂q_k prioritizes which step to upgrade first.
+- **Tier pricing utilities** — built-in `TIER_A`, `TIER_B`, `TIER_C` constants covering frontier, mid-range, and lightweight models. Pass your own `TierCost(tier_id, cost_per_1k_tokens)` to use current pricing.
+- **`StepExecutor` protocol** — plug in real LLM calls or a `Mock` for testing, no SDK dependency.
+
+---
+
+## Installation
+
+```bash
+pip install agent-tco
+```
+
+Built for Python 3.12 or above.
+
+---
+
+## Quick Start
+
+### 1. Define configurations and run the grid search
+
+```python
+from agent_tco import (
+    CostParams, GridSearch,
+    step_config_from_tier, WorkflowConfig,
+    TIER_A, TIER_B, TIER_C,
+)
+
+cost_params = CostParams(
+    labor_rate=60.0,
+    review_time_hours=0.25,
+    manual_process_cost=50.0,
+)
+
+# j3: all Tier C
+j3 = WorkflowConfig("j3", [
+    step_config_from_tier(k, TIER_C, acr=0.78 if k == 2 else 0.92, hitl_prob=0.20 if k == 4 else 0.05)
+    for k in range(1, 6)
+])
+
+# j9: Tier B at step 2 (the bottleneck), Tier C elsewhere
+j9 = WorkflowConfig("j9", [
+    step_config_from_tier(k, TIER_B if k == 2 else TIER_C,
+                          acr=0.88 if k == 2 else 0.92,
+                          hitl_prob=0.08 if k == 2 else (0.20 if k == 4 else 0.05))
+    for k in range(1, 6)
+])
+
+gs = GridSearch(cost_params=cost_params, n_runs=300, seed=42)
+results = gs.evaluate_all([j3, j9])
+for r in results:
+    print(f"{r.config.config_id}: ACR={r.acr:.2f}  E[TCO]=${r.expected_tco:.2f}  M=${r.executive_metric:.2f}")
+```
+
+### 2. Compute the Pareto frontier and identify the recommended configuration
+
+```python
+from agent_tco import pareto_frontier, rank_by_executive_metric
+
+frontier = pareto_frontier(results)
+ranked = rank_by_executive_metric(frontier)
+best = ranked[0]
+print(f"Recommended: {best.config.config_id} — ${best.executive_metric:.2f} per autonomous outcome")
+```
+
+---
+
+## Interpreting the Executive Metric
+
+The executive metric `M = E[TCO] / ACR` is the expected cost per autonomously delivered outcome. When `M < C_manual` (the cost of the manual process), the agent is economically viable at this configuration. If `M > C_manual` for every configuration on the Pareto frontier, no configuration is cost-effective and the business case does not hold.
+
+---
+
+## Using a real LLM executor
+
+The `StepExecutor` protocol accepts any callable `(step, context) -> (completed, hitl_triggered)`:
+
+```python
+from agent_tco import WorkflowSimulator, CostParams
+
+def my_llm_executor(step, context):
+    # call your LLM here
+    completed = run_llm_step(step, context)
+    hitl = should_escalate(step, context)
+    return completed, hitl
+
+sim = WorkflowSimulator(cost_params=cost_params, executor=my_llm_executor)
+run = sim.simulate_run(j9)
+print(run.scenario, run.total_cost)
+```
+
+In tests, replace `my_llm_executor` with `Mock(return_value=(True, False))`.
+
+---
+
+## API Reference
+
+See [API.md](API.md).
+
+---
+
+## License
+
+MIT License. See [LICENSE](LICENSE).

agent_tco-0.0.1/README.md

@@ -0,0 +1,114 @@
+# agent-tco
+
+**agent-tco** is a Python library for Pareto-optimal configuration of agentic workflows.
+
+---
+
+## Features
+
+- **Workflow simulator** with six-class scenario taxonomy (i1–i6): happy path, retry-resolved, HITL success, HITL partial, compound recovery, aborted.
+- **Grid search** — evaluates all candidate configurations with Beta credible intervals on scenario probabilities.
+- **Pareto frontier** — identifies non-dominated configurations on the ACR vs. E[TCO] plane.
+- **Critical step sensitivity analysis** — Proposition 1: ∂ACR/∂q_k prioritizes which step to upgrade first.
+- **Tier pricing utilities** — built-in `TIER_A`, `TIER_B`, `TIER_C` constants covering frontier, mid-range, and lightweight models. Pass your own `TierCost(tier_id, cost_per_1k_tokens)` to use current pricing.
+- **`StepExecutor` protocol** — plug in real LLM calls or a `Mock` for testing, no SDK dependency.
+
+---
+
+## Installation
+
+```bash
+pip install agent-tco
+```
+
+Built for Python 3.12 or above.
+
+---
+
+## Quick Start
+
+### 1. Define configurations and run the grid search
+
+```python
+from agent_tco import (
+    CostParams, GridSearch,
+    step_config_from_tier, WorkflowConfig,
+    TIER_A, TIER_B, TIER_C,
+)
+
+cost_params = CostParams(
+    labor_rate=60.0,
+    review_time_hours=0.25,
+    manual_process_cost=50.0,
+)
+
+# j3: all Tier C
+j3 = WorkflowConfig("j3", [
+    step_config_from_tier(k, TIER_C, acr=0.78 if k == 2 else 0.92, hitl_prob=0.20 if k == 4 else 0.05)
+    for k in range(1, 6)
+])
+
+# j9: Tier B at step 2 (the bottleneck), Tier C elsewhere
+j9 = WorkflowConfig("j9", [
+    step_config_from_tier(k, TIER_B if k == 2 else TIER_C,
+                          acr=0.88 if k == 2 else 0.92,
+                          hitl_prob=0.08 if k == 2 else (0.20 if k == 4 else 0.05))
+    for k in range(1, 6)
+])
+
+gs = GridSearch(cost_params=cost_params, n_runs=300, seed=42)
+results = gs.evaluate_all([j3, j9])
+for r in results:
+    print(f"{r.config.config_id}: ACR={r.acr:.2f}  E[TCO]=${r.expected_tco:.2f}  M=${r.executive_metric:.2f}")
+```
+
+### 2. Compute the Pareto frontier and identify the recommended configuration
+
+```python
+from agent_tco import pareto_frontier, rank_by_executive_metric
+
+frontier = pareto_frontier(results)
+ranked = rank_by_executive_metric(frontier)
+best = ranked[0]
+print(f"Recommended: {best.config.config_id} — ${best.executive_metric:.2f} per autonomous outcome")
+```
+
+---
+
+## Interpreting the Executive Metric
+
+The executive metric `M = E[TCO] / ACR` is the expected cost per autonomously delivered outcome. When `M < C_manual` (the cost of the manual process), the agent is economically viable at this configuration. If `M > C_manual` for every configuration on the Pareto frontier, no configuration is cost-effective and the business case does not hold.
+
+---
+
+## Using a real LLM executor
+
+The `StepExecutor` protocol accepts any callable `(step, context) -> (completed, hitl_triggered)`:
+
+```python
+from agent_tco import WorkflowSimulator, CostParams
+
+def my_llm_executor(step, context):
+    # call your LLM here
+    completed = run_llm_step(step, context)
+    hitl = should_escalate(step, context)
+    return completed, hitl
+
+sim = WorkflowSimulator(cost_params=cost_params, executor=my_llm_executor)
+run = sim.simulate_run(j9)
+print(run.scenario, run.total_cost)
+```
+
+In tests, replace `my_llm_executor` with `Mock(return_value=(True, False))`.
+
+---
+
+## API Reference
+
+See [API.md](API.md).
+
+---
+
+## License
+
+MIT License. See [LICENSE](LICENSE).

agent_tco-0.0.1/pyproject.toml

@@ -0,0 +1,26 @@
+[project]
+name = "agent-tco"
+version = "0.0.1"
+license = { file = "LICENSE" }
+description = "agent-tco: Pareto-optimal configuration of agentic workflows."
+readme = "README.md"
+authors = [
+    { name = "Chinmay Kakatkar" }
+]
+requires-python = ">=3.12"
+dependencies = [
+    "scipy>=1.10",
+]
+
+[project.urls]
+Homepage = "https://github.com/ckstash/agent-tco"
+
+[build-system]
+requires = ["uv_build>=0.8.11,<0.9.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "pydoc-markdown>=4.8.2",
+    "pytest>=9.1.1",
+]

agent_tco-0.0.1/src/agent_tco/__init__.py

@@ -0,0 +1,51 @@
+from __future__ import annotations
+
+from agent_tco.executor import StepExecutor
+from agent_tco.grid import GridResult, GridSearch, ScenarioStats
+from agent_tco.model import CostParams, Scenario, SimulationRun, StepConfig, WorkflowConfig
+from agent_tco.pareto import (
+    dominates,
+    executive_metric,
+    pareto_frontier,
+    rank_by_executive_metric,
+)
+from agent_tco.pricing import (
+    TIER_A,
+    TIER_B,
+    TIER_C,
+    TierCost,
+    inference_cost,
+    step_config_from_tier,
+)
+from agent_tco.sensitivity import (
+    critical_step_sensitivity,
+    marginal_acr_gain,
+    sample_complexity,
+)
+from agent_tco.simulator import WorkflowSimulator
+
+__all__ = [
+    "Scenario",
+    "StepConfig",
+    "WorkflowConfig",
+    "CostParams",
+    "SimulationRun",
+    "StepExecutor",
+    "WorkflowSimulator",
+    "ScenarioStats",
+    "GridResult",
+    "GridSearch",
+    "dominates",
+    "pareto_frontier",
+    "executive_metric",
+    "rank_by_executive_metric",
+    "marginal_acr_gain",
+    "critical_step_sensitivity",
+    "sample_complexity",
+    "TierCost",
+    "TIER_A",
+    "TIER_B",
+    "TIER_C",
+    "inference_cost",
+    "step_config_from_tier",
+]

agent_tco-0.0.1/src/agent_tco/executor.py

@@ -0,0 +1,25 @@
+from __future__ import annotations
+
+from typing import Protocol
+
+from agent_tco.model import StepConfig
+
+
+class StepExecutor(Protocol):
+    """Protocol for executing one LLM step in a workflow.
+
+    Any callable matching (step: StepConfig, context: dict) -> tuple[bool, bool]
+    satisfies this protocol. The return value is (completed, hitl_triggered).
+
+    In production, implement this with a real LLM call.
+    In tests, pass a lambda or unittest.mock.Mock to control outcomes without
+    any network call.
+    """
+
+    def __call__(
+        self,
+        step: StepConfig,
+        context: dict,
+    ) -> tuple[bool, bool]:
+        """Execute the step and return (completed, hitl_triggered)."""
+        ...

agent_tco-0.0.1/src/agent_tco/grid.py

@@ -0,0 +1,101 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+
+from scipy.stats import beta as beta_dist
+
+from agent_tco.model import CostParams, Scenario, WorkflowConfig
+from agent_tco.simulator import WorkflowSimulator
+
+
+@dataclass
+class ScenarioStats:
+    """Statistics for one scenario class across N simulation runs."""
+
+    scenario: Scenario
+    probability: float   # empirical P_i = count_i / N
+    ci_lower: float      # 95% Beta credible interval lower bound
+    ci_upper: float      # 95% Beta credible interval upper bound
+    mean_cost: float     # empirical mean cost within this scenario (0.0 if no runs)
+
+
+@dataclass
+class GridResult:
+    """Aggregated results for one WorkflowConfig after N simulation runs."""
+
+    config: WorkflowConfig
+    n_runs: int
+    scenario_stats: dict[Scenario, ScenarioStats]
+    acr: float            # P(i1) + P(i2)
+    expected_tco: float   # sum_i P_i * mean_cost_i
+    executive_metric: float  # expected_tco / acr; inf if acr == 0
+    pareto_dominated: bool = False
+
+
+def _beta_ci(k: int, n: int) -> tuple[float, float]:
+    """Return the 95% Beta posterior credible interval for k successes in n trials."""
+    lower = float(beta_dist.ppf(0.025, k + 1, n - k + 1))
+    upper = float(beta_dist.ppf(0.975, k + 1, n - k + 1))
+    return lower, upper
+
+
+class GridSearch:
+    """Evaluate agentic workflow configurations over N simulation runs each."""
+
+    def __init__(
+        self,
+        cost_params: CostParams,
+        n_runs: int = 300,
+        seed: int = 42,
+    ) -> None:
+        """Initialize the grid search with cost parameters and simulation budget."""
+        self._cost_params = cost_params
+        self._n_runs = n_runs
+        self._seed = seed
+
+    def evaluate(self, config: WorkflowConfig) -> GridResult:
+        """Run n_runs simulations for one config and return a GridResult."""
+        sim = WorkflowSimulator(cost_params=self._cost_params, seed=self._seed)
+        runs = [sim.simulate_run(config, run_index=i) for i in range(self._n_runs)]
+        return self._aggregate(config, runs)
+
+    def evaluate_all(self, configs: list[WorkflowConfig]) -> list[GridResult]:
+        """Evaluate all configs sequentially and return one GridResult per config."""
+        return [self.evaluate(c) for c in configs]
+
+    def _aggregate(self, config: WorkflowConfig, runs: list) -> GridResult:
+        """Compute scenario statistics and composite metrics from a list of SimulationRun."""
+        n = len(runs)
+        scenario_stats: dict[Scenario, ScenarioStats] = {}
+
+        for scenario in Scenario:
+            matching = [r for r in runs if r.scenario == scenario]
+            k = len(matching)
+            probability = k / n
+            ci_lower, ci_upper = _beta_ci(k, n)
+            mean_cost = sum(r.total_cost for r in matching) / k if k > 0 else 0.0
+            scenario_stats[scenario] = ScenarioStats(
+                scenario=scenario,
+                probability=probability,
+                ci_lower=ci_lower,
+                ci_upper=ci_upper,
+                mean_cost=mean_cost,
+            )
+
+        acr = (
+            scenario_stats[Scenario.HAPPY_PATH].probability
+            + scenario_stats[Scenario.RETRY_RESOLVED].probability
+        )
+        expected_tco = sum(
+            s.probability * s.mean_cost for s in scenario_stats.values()
+        )
+        exec_metric = expected_tco / acr if acr > 0 else float("inf")
+
+        return GridResult(
+            config=config,
+            n_runs=n,
+            scenario_stats=scenario_stats,
+            acr=acr,
+            expected_tco=expected_tco,
+            executive_metric=exec_metric,
+        )

agent_tco-0.0.1/src/agent_tco/model.py

@@ -0,0 +1,63 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from enum import Enum
+
+
+class Scenario(Enum):
+    """The six mutually exclusive trajectory outcome classes."""
+
+    HAPPY_PATH = "i1"          # r=0, h=0, s=success
+    RETRY_RESOLVED = "i2"      # r>=1, h=0, s=success
+    HITL_SUCCESS = "i3"        # h=1, r=0, s=success
+    HITL_PARTIAL = "i4"        # h=1, s=partial
+    COMPOUND_RECOVERY = "i5"   # r>=1, h=1, s=success
+    ABORTED = "i6"             # s=aborted (any r, h)
+
+
+@dataclass
+class StepConfig:
+    """Per-step configuration for one LLM touchpoint in a workflow."""
+
+    step_index: int        # 1-based position in the workflow
+    model_tier: str        # informational label, e.g. "A", "B", "C"
+    acr: float             # per-step autonomous completion rate in [0, 1]
+    hitl_prob: float       # probability of HITL trigger in [0, 1]
+    inference_cost: float  # cost per LLM call for this step (dollars)
+
+
+@dataclass
+class WorkflowConfig:
+    """A complete workflow configuration: one StepConfig per touchpoint."""
+
+    config_id: str
+    steps: list[StepConfig]
+
+    @property
+    def n_steps(self) -> int:
+        """Return the number of steps in this configuration."""
+        return len(self.steps)
+
+
+@dataclass
+class CostParams:
+    """Cost parameters that are independent of the workflow configuration."""
+
+    labor_rate: float           # dollars per hour
+    review_time_hours: float    # hours per HITL review event
+    manual_process_cost: float  # opportunity cost applied when a run is aborted
+
+    @property
+    def hitl_cost_per_event(self) -> float:
+        """Return the cost of one HITL review event in dollars."""
+        return self.labor_rate * self.review_time_hours
+
+
+@dataclass
+class SimulationRun:
+    """Result of one simulated workflow run."""
+
+    scenario: Scenario
+    total_cost: float
+    retry_count: int
+    hitl_triggered: bool

agent_tco-0.0.1/src/agent_tco/pareto.py

@@ -0,0 +1,45 @@
+from __future__ import annotations
+
+from agent_tco.grid import GridResult
+
+
+def dominates(a: GridResult, b: GridResult) -> bool:
+    """Return True if a Pareto-dominates b.
+
+    a dominates b when a.acr >= b.acr and a.expected_tco <= b.expected_tco,
+    with at least one strict inequality.
+    """
+    better_acr = a.acr >= b.acr
+    better_tco = a.expected_tco <= b.expected_tco
+    strictly_better = (a.acr > b.acr) or (a.expected_tco < b.expected_tco)
+    return better_acr and better_tco and strictly_better
+
+
+def pareto_frontier(results: list[GridResult]) -> list[GridResult]:
+    """Return the non-dominated subset of results.
+
+    Sets pareto_dominated=True on every dominated GridResult in-place.
+    """
+    for r in results:
+        r.pareto_dominated = False
+
+    frontier = []
+    for candidate in results:
+        dominated = any(dominates(other, candidate) for other in results if other is not candidate)
+        if dominated:
+            candidate.pareto_dominated = True
+        else:
+            frontier.append(candidate)
+    return frontier
+
+
+def executive_metric(expected_tco: float, acr: float) -> float:
+    """Return expected_tco / acr; float('inf') if acr == 0."""
+    if acr == 0:
+        return float("inf")
+    return expected_tco / acr
+
+
+def rank_by_executive_metric(results: list[GridResult]) -> list[GridResult]:
+    """Return results sorted ascending by executive_metric (lowest cost-per-outcome first)."""
+    return sorted(results, key=lambda r: r.executive_metric)

agent_tco-0.0.1/src/agent_tco/pricing.py

@@ -0,0 +1,41 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from agent_tco.model import StepConfig
+
+
+@dataclass
+class TierCost:
+    """Per-1K-output-token cost for one model tier."""
+
+    tier_id: str
+    cost_per_1k_tokens: float  # dollars
+
+
+# Built-in tier cost defaults. Override with your own TierCost if pricing has changed.
+TIER_A = TierCost("A", 0.003)    # frontier model (e.g. GPT-4o / Claude Sonnet class)
+TIER_B = TierCost("B", 0.0008)   # mid-range model (e.g. GPT-4o Mini / Claude Haiku class)
+TIER_C = TierCost("C", 0.00015)  # lightweight model
+
+
+def inference_cost(output_tokens: int, tier: TierCost) -> float:
+    """Return the inference cost in dollars for output_tokens at tier pricing."""
+    return (output_tokens / 1000) * tier.cost_per_1k_tokens
+
+
+def step_config_from_tier(
+    step_index: int,
+    tier: TierCost,
+    acr: float,
+    hitl_prob: float,
+    avg_output_tokens: int = 500,
+) -> StepConfig:
+    """Construct a StepConfig from a TierCost and estimated average token usage."""
+    return StepConfig(
+        step_index=step_index,
+        model_tier=tier.tier_id,
+        acr=acr,
+        hitl_prob=hitl_prob,
+        inference_cost=inference_cost(avg_output_tokens, tier),
+    )

agent_tco-0.0.1/src/agent_tco/sensitivity.py

@@ -0,0 +1,55 @@
+from __future__ import annotations
+
+import math
+
+from agent_tco.model import WorkflowConfig
+
+
+def marginal_acr_gain(config: WorkflowConfig, step_index: int) -> float:
+    """Return the marginal ACR gain from improving step step_index by one unit.
+
+    ∂ACR/∂q_k = product of q_j for all j != step_index.
+    step_index is 1-based. Uses the no-retry independence assumption.
+    """
+    indices = [s.step_index for s in config.steps]
+    if step_index not in indices:
+        raise ValueError(
+            f"step_index {step_index} not found in config steps {indices}"
+        )
+    result = 1.0
+    for step in config.steps:
+        if step.step_index != step_index:
+            result *= step.acr
+    return result
+
+
+def critical_step_sensitivity(config: WorkflowConfig) -> list[tuple[int, float]]:
+    """Return (step_index, marginal_acr_gain) pairs sorted descending by gain.
+
+    The first entry is the bottleneck step — the highest-priority upgrade target.
+    """
+    gains = [
+        (step.step_index, marginal_acr_gain(config, step.step_index))
+        for step in config.steps
+    ]
+    return sorted(gains, key=lambda x: x[1], reverse=True)
+
+
+def sample_complexity(
+    c_max: float,
+    epsilon: float,
+    delta: float = 0.05,
+) -> int:
+    """Return the minimum N for E[TCO] estimation within epsilon at confidence 1-delta.
+
+    Uses the Hoeffding bound:
+    N >= ceil(c_max^2 * log(6 / delta) / (2 * epsilon^2))
+    """
+    if epsilon <= 0:
+        raise ValueError(f"epsilon must be positive; got {epsilon}")
+    if not (0 < delta < 1):
+        raise ValueError(f"delta must be in (0, 1); got {delta}")
+    if c_max <= 0:
+        raise ValueError(f"c_max must be positive; got {c_max}")
+    n = (c_max ** 2 * math.log(6 / delta)) / (2 * epsilon ** 2)
+    return math.ceil(n)

agent_tco-0.0.1/src/agent_tco/simulator.py

@@ -0,0 +1,105 @@
+from __future__ import annotations
+
+import random
+from typing import Callable
+
+from agent_tco.model import (
+    CostParams,
+    Scenario,
+    SimulationRun,
+    StepConfig,
+    WorkflowConfig,
+)
+
+
+class WorkflowSimulator:
+    """Simulate one run of a workflow under a given configuration.
+
+    Implements the single-retry model: if a step does not complete on the first
+    attempt, one retry is made. If the retry also fails, the run is aborted
+    (scenario i6) and the opportunity cost is applied.
+
+    Pass an optional executor callable to drive steps with real LLM calls instead
+    of Bernoulli sampling. The executor must match StepExecutor protocol:
+    (step: StepConfig, context: dict) -> tuple[bool, bool] (completed, hitl_triggered).
+    """
+
+    def __init__(
+        self,
+        cost_params: CostParams,
+        seed: int = 42,
+        executor: Callable | None = None,
+    ) -> None:
+        """Initialize the simulator with cost parameters and optional LLM executor."""
+        self._cost_params = cost_params
+        self._seed = seed
+        self._executor = executor
+
+    def simulate_run(
+        self,
+        config: WorkflowConfig,
+        run_index: int = 0,
+    ) -> SimulationRun:
+        """Simulate one workflow run and return the classified outcome."""
+        rng = random.Random(self._seed + run_index)
+        cost = 0.0
+        retry_count = 0
+        hitl_triggered = False
+        terminal = "success"
+        context: dict = {}
+
+        for step in config.steps:
+            cost += step.inference_cost  # first-attempt inference cost
+
+            if self._executor is not None:
+                completed, hitl = self._executor(step, context)
+            else:
+                hitl = rng.random() < step.hitl_prob
+                completed = rng.random() < step.acr
+
+            if hitl:
+                hitl_triggered = True
+                cost += self._cost_params.hitl_cost_per_event
+
+            if not completed:
+                # Single retry
+                retry_count += 1
+                cost += step.inference_cost  # retry inference cost
+                if self._executor is not None:
+                    completed, _ = self._executor(step, context)
+                else:
+                    completed = rng.random() < step.acr
+
+                if not completed:
+                    cost += self._cost_params.manual_process_cost
+                    terminal = "aborted"
+                    break
+
+        scenario = self.classify_scenario(retry_count, hitl_triggered, terminal)
+        return SimulationRun(
+            scenario=scenario,
+            total_cost=cost,
+            retry_count=retry_count,
+            hitl_triggered=hitl_triggered,
+        )
+
+    def classify_scenario(
+        self,
+        retry_count: int,
+        hitl_triggered: bool,
+        terminal_state: str,
+    ) -> Scenario:
+        """Classify a run outcome into one of the six scenario classes."""
+        if terminal_state == "aborted":
+            return Scenario.ABORTED
+        if terminal_state == "partial":
+            return Scenario.HITL_PARTIAL  # h=1 implied when partial
+        # terminal_state == "success"
+        if not hitl_triggered and retry_count == 0:
+            return Scenario.HAPPY_PATH
+        if not hitl_triggered and retry_count >= 1:
+            return Scenario.RETRY_RESOLVED
+        if hitl_triggered and retry_count == 0:
+            return Scenario.HITL_SUCCESS
+        # hitl_triggered and retry_count >= 1
+        return Scenario.COMPOUND_RECOVERY