rote-cli 0.6.0__tar.gz → 0.7.0__tar.gz

{rote_cli-0.6.0 → rote_cli-0.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rote-cli
-Version: 0.6.0
+Version: 0.7.0
 Summary: Graduate fuzzy AI skills into deterministic, reliable workflows
 Project-URL: Homepage, https://github.com/trevhud/rote
 Project-URL: Repository, https://github.com/trevhud/rote
@@ -326,17 +326,16 @@ subscription path.
 
 ## Status
 
-`rote` is **pre-1.0**. The end-to-end flow works on the BDR example, and
-all six adapters are validated by a `slow`-marked e2e suite that runs the
-emitted code against real runtimes (DBOS over SQLite, Temporal's
-time-skipping server, the TypeScript targets compiled with `tsc --noEmit`
-and driven through both HITL gates on live dev servers, the plain-Python
-script as a subprocess, and the MCP server over real stdio). The fast
-suite (`pytest tests/`) makes no real API calls; the integration suite is
-`pytest tests/ -m slow`.
-
-Known gaps: the `codex` driver is a stub (`is_available` works, `run`
-isn't implemented), the extracted modules are `NotImplementedError` stubs
+`rote` is **pre-1.0**. The end-to-end flow works on the BDR example. The
+fast suite (`pytest tests/`) makes no real API calls and is what CI runs
+on every push, alongside a Python e2e (DBOS over SQLite + the MCP server
+over real stdio). Each adapter also has a `slow`-marked e2e that runs its
+emitted code against the real runtime (Temporal's time-skipping server,
+the TypeScript targets via `tsc --noEmit` and live dev servers, the
+plain-Python subprocess); those need a Node toolchain / Docker, so they
+run locally with `pytest tests/ -m slow`, not in CI.
+
+Known gaps: the extracted modules are `NotImplementedError` stubs
 you fill in with real API-client code, a Restate adapter is planned, and
 `fan_out` nodes currently receive the whole upstream list in one
 invocation (per-element dispatch is a planned enhancement). Published on
@@ -357,7 +356,10 @@ rote/
 │   ├── graduator/         orchestrator + drivers/ (claude · codex · anthropic_api)
 │   └── adapters/          dbos · temporal · python · cloudflare · dbos_ts · inngest
 │                          (+ _common / _py_common / _ts_common emit helpers)
-├── examples/bdr-outreach/ source skill · hand-drafted IR baseline · run snapshots
+├── examples/
+│   ├── bdr-outreach/      canonical: all 5 node kinds · IR baseline · run snapshots
+│   ├── ops-report/        100% roteness: zero LLM nodes + a HITL gate
+│   └── deal-monitor/      data-heavy: parallel waves · fan-out judges · template render
 └── tests/                 fast + slow suites (pytest -m slow)
 ```
 
@@ -377,6 +379,12 @@ rote/
   heuristics, IR schema, LLM-judge extraction)
 - [`examples/bdr-outreach/`](examples/bdr-outreach/) — the canonical
   skill, its ground-truth IR, and snapshotted real graduator runs
+- [`examples/ops-report/`](examples/ops-report/) — the 100%-roteness
+  archetype: every step deterministic, one durable HITL gate, zero LLM
+  nodes after graduation
+- [`examples/deal-monitor/`](examples/deal-monitor/) — the data-heavy
+  archetype: parallel entry waves, fan-out judges, and a template render
+  replacing per-run LLM-generated HTML
 
 ---
 
@@ -384,19 +392,17 @@ rote/
 
 In rough priority order:
 
-1. **`CodexDriver` implementation** — same shape as `ClaudeDriver` but
-   spawning `codex exec`; unlocks ChatGPT subscribers.
-2. **Re-graduate BDR end-to-end with `signature_spec`** — the bundled IR
+1. **Re-graduate BDR end-to-end with `signature_spec`** — the bundled IR
    was hand-extended with structured schemas; the rubric now teaches the
    field, but no real run has produced one yet.
-3. **Pre-filter as a `pure_function` node** — today hard thresholds are
+2. **Pre-filter as a `pure_function` node** — today hard thresholds are
    lifted into a judge's `forward()`, which works for Temporal but not
    Cloudflare; a separate node makes the short-circuit uniform.
-4. **More example skills** — BDR is one shape; research-heavy,
+3. **More example skills** — BDR is one shape; research-heavy,
    retrieval-heavy, and code-review skills stress the IR differently.
-5. **`fan_out` per-element dispatch** — currently the whole upstream list
+4. **`fan_out` per-element dispatch** — currently the whole upstream list
    arrives in one invocation.
-6. **The graduator graduating itself** — `rote-graduate` is a SKILL.md;
+5. **The graduator graduating itself** — `rote-graduate` is a SKILL.md;
    pointing `rote graduate` at it should crystallize its rubric-grade
    pieces and leave only the genuinely fuzzy judgments in the loop.
 

{rote_cli-0.6.0 → rote_cli-0.7.0}/README.md

@@ -280,17 +280,16 @@ subscription path.
 
 ## Status
 
-`rote` is **pre-1.0**. The end-to-end flow works on the BDR example, and
-all six adapters are validated by a `slow`-marked e2e suite that runs the
-emitted code against real runtimes (DBOS over SQLite, Temporal's
-time-skipping server, the TypeScript targets compiled with `tsc --noEmit`
-and driven through both HITL gates on live dev servers, the plain-Python
-script as a subprocess, and the MCP server over real stdio). The fast
-suite (`pytest tests/`) makes no real API calls; the integration suite is
-`pytest tests/ -m slow`.
-
-Known gaps: the `codex` driver is a stub (`is_available` works, `run`
-isn't implemented), the extracted modules are `NotImplementedError` stubs
+`rote` is **pre-1.0**. The end-to-end flow works on the BDR example. The
+fast suite (`pytest tests/`) makes no real API calls and is what CI runs
+on every push, alongside a Python e2e (DBOS over SQLite + the MCP server
+over real stdio). Each adapter also has a `slow`-marked e2e that runs its
+emitted code against the real runtime (Temporal's time-skipping server,
+the TypeScript targets via `tsc --noEmit` and live dev servers, the
+plain-Python subprocess); those need a Node toolchain / Docker, so they
+run locally with `pytest tests/ -m slow`, not in CI.
+
+Known gaps: the extracted modules are `NotImplementedError` stubs
 you fill in with real API-client code, a Restate adapter is planned, and
 `fan_out` nodes currently receive the whole upstream list in one
 invocation (per-element dispatch is a planned enhancement). Published on
@@ -311,7 +310,10 @@ rote/
 │   ├── graduator/         orchestrator + drivers/ (claude · codex · anthropic_api)
 │   └── adapters/          dbos · temporal · python · cloudflare · dbos_ts · inngest
 │                          (+ _common / _py_common / _ts_common emit helpers)
-├── examples/bdr-outreach/ source skill · hand-drafted IR baseline · run snapshots
+├── examples/
+│   ├── bdr-outreach/      canonical: all 5 node kinds · IR baseline · run snapshots
+│   ├── ops-report/        100% roteness: zero LLM nodes + a HITL gate
+│   └── deal-monitor/      data-heavy: parallel waves · fan-out judges · template render
 └── tests/                 fast + slow suites (pytest -m slow)
 ```
 
@@ -331,6 +333,12 @@ rote/
   heuristics, IR schema, LLM-judge extraction)
 - [`examples/bdr-outreach/`](examples/bdr-outreach/) — the canonical
   skill, its ground-truth IR, and snapshotted real graduator runs
+- [`examples/ops-report/`](examples/ops-report/) — the 100%-roteness
+  archetype: every step deterministic, one durable HITL gate, zero LLM
+  nodes after graduation
+- [`examples/deal-monitor/`](examples/deal-monitor/) — the data-heavy
+  archetype: parallel entry waves, fan-out judges, and a template render
+  replacing per-run LLM-generated HTML
 
 ---
 
@@ -338,19 +346,17 @@ rote/
 
 In rough priority order:
 
-1. **`CodexDriver` implementation** — same shape as `ClaudeDriver` but
-   spawning `codex exec`; unlocks ChatGPT subscribers.
-2. **Re-graduate BDR end-to-end with `signature_spec`** — the bundled IR
+1. **Re-graduate BDR end-to-end with `signature_spec`** — the bundled IR
    was hand-extended with structured schemas; the rubric now teaches the
    field, but no real run has produced one yet.
-3. **Pre-filter as a `pure_function` node** — today hard thresholds are
+2. **Pre-filter as a `pure_function` node** — today hard thresholds are
    lifted into a judge's `forward()`, which works for Temporal but not
    Cloudflare; a separate node makes the short-circuit uniform.
-4. **More example skills** — BDR is one shape; research-heavy,
+3. **More example skills** — BDR is one shape; research-heavy,
    retrieval-heavy, and code-review skills stress the IR differently.
-5. **`fan_out` per-element dispatch** — currently the whole upstream list
+4. **`fan_out` per-element dispatch** — currently the whole upstream list
    arrives in one invocation.
-6. **The graduator graduating itself** — `rote-graduate` is a SKILL.md;
+5. **The graduator graduating itself** — `rote-graduate` is a SKILL.md;
    pointing `rote graduate` at it should crystallize its rubric-grade
    pieces and leave only the genuinely fuzzy judgments in the loop.
 

{rote_cli-0.6.0 → rote_cli-0.7.0}/src/rote/__init__.py

@@ -1,3 +1,3 @@
 """rote — graduate fuzzy AI skills into deterministic workflows."""
 
-__version__ = "0.6.0"
+__version__ = "0.7.0"

{rote_cli-0.6.0 → rote_cli-0.7.0}/src/rote/adapters/_common.py

@@ -223,13 +223,17 @@ def _pipeline_hash(pipeline: Pipeline) -> str:
     # Hash the full validated contents: node wiring, inputs, retries, and
     # edges all participate, so any regeneration that changes behavior gets
     # a new workflow type — name/version/counts alone miss rewires.
-    # Provenance (``Node.source``) is excluded: it's tooling metadata, and
-    # annotating where a node came from must not re-version in-flight
-    # workflows whose behavior didn't change.
+    # Provenance is excluded — it's tooling metadata, and annotating where
+    # a pipeline came from must not re-version in-flight workflows whose
+    # behavior didn't change. That covers ``Node.source`` (which SKILL.md
+    # section produced a node) and ``source_skill`` (where the skill bundle
+    # lived): the latter is a filesystem path the graduator re-points per
+    # output location, so hashing it would mint a new workflow type on
+    # every re-graduation to a different directory.
     payload = pipeline.model_dump_json(
         by_alias=True,
         exclude_none=True,
-        exclude={"nodes": {"__all__": {"source"}}},
+        exclude={"source_skill": True, "nodes": {"__all__": {"source"}}},
     )
     return hashlib.sha256(payload.encode()).hexdigest()[:8]
 

{rote_cli-0.6.0 → rote_cli-0.7.0}/src/rote/cli.py

@@ -28,7 +28,7 @@ import asyncio
 import sys
 from collections.abc import Sequence
 from pathlib import Path
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, Any
 
 from rote import __version__
 from rote.adapters import ADAPTERS, get_adapter
@@ -36,6 +36,7 @@ from rote.graduator import Graduator, GraduatorError
 from rote.ir import Pipeline, load_pipeline
 
 if TYPE_CHECKING:
+    from rote.eval.priors import Priors
     from rote.eval.scorecard import Scorecard
 
 # ───────── Subcommand: emit ─────────
@@ -377,9 +378,201 @@ def _cmd_serve(args: argparse.Namespace) -> int:
     return 0
 
 
+def _build_analysis(
+    pipeline: Pipeline,
+    driver_name: str,
+    driver_metadata: dict[str, object],
+    skill_path: Path,
+) -> dict[str, object]:
+    """Derive a structural report from a graduated pipeline's IR.
+
+    Pure function of the IR — no model, no network. Step counting mirrors
+    the eval scorecard exactly (top-level execution-wave nodes; loop_body
+    sub-nodes are costed inside their parent loop, so they're excluded
+    from the top-level count) so ``analyze``'s roteness equals ``eval``'s.
+    """
+    from rote.adapters import ADAPTERS
+    from rote.adapters._common import _execution_waves
+    from rote.ir import NodeKind
+
+    top_level = [n for wave in _execution_waves(pipeline) for n in wave]
+    total = len(top_level)
+
+    by_kind = {k.value: 0 for k in NodeKind}
+    for n in top_level:
+        by_kind[n.kind.value] += 1
+
+    sampled = by_kind[NodeKind.LLM_JUDGE.value] + by_kind[NodeKind.AGENT_LOOP.value]
+    deterministic = total - sampled
+    roteness = (deterministic / total) if total else 0.0
+
+    loop_body_ids = sorted({b for n in pipeline.nodes if n.loop_body for b in n.loop_body})
+
+    untargetable: dict[str, str] = {}
+    if pipeline.requires_durable_execution:
+        untargetable["python"] = (
+            "pipeline parks on a human gate (hitl_gate); needs durable execution"
+        )
+    targetable = [name for name in sorted(ADAPTERS) if name not in untargetable]
+
+    return {
+        "pipeline": pipeline.name,
+        "version": pipeline.version,
+        "description": pipeline.description,
+        "driver": driver_name,
+        "driver_metadata": driver_metadata,
+        "source_skill": str(skill_path),
+        "nodes": {
+            "total": total,
+            "by_kind": by_kind,
+            "loop_body_subnodes": loop_body_ids,
+        },
+        "roteness": roteness,
+        "deterministic_steps": deterministic,
+        "sampled_steps": sampled,
+        "mandatory": [n.id for n in top_level if n.mandatory],
+        "hitl_gates": [
+            {"id": n.id, "signal": n.signal} for n in top_level if n.kind is NodeKind.HITL_GATE
+        ],
+        "agent_loops": [
+            {
+                "id": n.id,
+                "max_iterations": (n.termination.max_iterations if n.termination else None),
+            }
+            for n in top_level
+            if n.kind is NodeKind.AGENT_LOOP
+        ],
+        "targetable_runtimes": targetable,
+        "untargetable_runtimes": untargetable,
+    }
+
+
+def _render_analysis_text(report: dict[str, object]) -> str:
+    """Render an :func:`_build_analysis` report as a scannable text block."""
+    nodes = report["nodes"]
+    assert isinstance(nodes, dict)
+    by_kind: dict[str, int] = nodes["by_kind"]
+    total = nodes["total"]
+
+    lines = [
+        f"rote analyze: {report['pipeline']} v{report['version']}",
+        f"  driver: {report['driver']}",
+        f"  source: {report['source_skill']}",
+    ]
+    description = report["description"]
+    if isinstance(description, str) and description.strip():
+        # Collapse internal whitespace/newlines to one line, then clip.
+        flat = " ".join(description.split())
+        lines.append(f"  {flat[:200] + '…' if len(flat) > 200 else flat}")
+
+    lines.append("")
+    lines.append(f"Nodes ({total} executed step{'s' if total != 1 else ''})")
+    # Stable, readable kind order; only show kinds that occur.
+    for kind in ("pure_function", "external_call", "llm_judge", "agent_loop", "hitl_gate"):
+        count = by_kind.get(kind, 0)
+        if count:
+            lines.append(f"  {kind:<15} {count:>2}  {'█' * count}")
+    subnodes = nodes["loop_body_subnodes"]
+    assert isinstance(subnodes, list)
+    if subnodes:
+        lines.append(f"  (+{len(subnodes)} loop-body sub-node(s), costed inside their loop)")
+
+    roteness = report["roteness"]
+    assert isinstance(roteness, float)
+    lines.append("")
+    lines.append(
+        f"Roteness: {roteness:.0%} deterministic "
+        f"({report['deterministic_steps']} of {total} steps run as code, not inference)"
+    )
+
+    mandatory = report["mandatory"]
+    assert isinstance(mandatory, list)
+    if mandatory:
+        lines.append(f"Mandatory checks (cannot be skipped): {', '.join(mandatory)}")
+
+    hitl = report["hitl_gates"]
+    assert isinstance(hitl, list)
+    if hitl:
+        gates = ", ".join(f"{g['id']} (signal: {g['signal']})" for g in hitl)
+        lines.append(f"HITL gates: {gates}")
+
+    loops = report["agent_loops"]
+    assert isinstance(loops, list)
+    if loops:
+        rendered = ", ".join(
+            f"{loop['id']} (max {loop['max_iterations']} iterations)"
+            if loop["max_iterations"] is not None
+            else str(loop["id"])
+            for loop in loops
+        )
+        lines.append(f"Agent loops: {rendered}")
+
+    targetable = report["targetable_runtimes"]
+    assert isinstance(targetable, list)
+    lines.append("")
+    lines.append(f"Targetable runtimes: {', '.join(targetable)}")
+    untargetable = report["untargetable_runtimes"]
+    assert isinstance(untargetable, dict)
+    for name, reason in untargetable.items():
+        lines.append(f"  {name}: unavailable — {reason}")
+
+    return "\n".join(lines)
+
+
 def _cmd_analyze(args: argparse.Namespace) -> int:
-    print("rote analyze: not yet implemented.", file=sys.stderr)
-    return 70
+    """Dry-run the graduator: report a skill's graduated shape, emit nothing.
+
+    The ``plan`` to ``graduate``'s ``apply`` — runs the same graduator
+    agent to produce a validated IR, then prints a structural report
+    instead of emitting runtime code. With ``--out`` the graduated IR +
+    stubs are kept (ready for a later ``rote emit``); without it, they're
+    produced in a temp dir and discarded after reporting.
+    """
+    import json
+    import tempfile
+    from contextlib import nullcontext
+
+    skill_path = Path(args.skill_path)
+    if not skill_path.is_dir():
+        print(f"error: skill path is not a directory: {skill_path}", file=sys.stderr)
+        return 2
+
+    graduator = Graduator(agent=args.agent, model=args.model)
+
+    out_ctx: Any = (
+        nullcontext(str(args.out))
+        if args.out
+        else tempfile.TemporaryDirectory(prefix="rote-analyze-")
+    )
+    with out_ctx as out_raw:
+        graduated_dir = Path(out_raw)
+        try:
+            result = asyncio.run(graduator.graduate(skill_path, graduated_dir))
+        except GraduatorError as e:
+            print(f"rote analyze: {e}", file=sys.stderr)
+            return 1
+        except KeyboardInterrupt:
+            print("rote analyze: interrupted", file=sys.stderr)
+            return 130
+
+        report = _build_analysis(
+            result.pipeline, result.driver_name, result.driver_metadata, skill_path
+        )
+        if args.json:
+            print(json.dumps(report, indent=2))
+        else:
+            print(_render_analysis_text(report))
+            if args.out:
+                print()
+                print(f"  graduated IR kept at: {graduated_dir / 'pipeline.yaml'}")
+                print(
+                    f"  emit a runtime with:  rote emit {graduated_dir / 'pipeline.yaml'} "
+                    f"--runtime dbos --out ./runtime"
+                )
+            else:
+                print()
+                print("  (report only — pass --out DIR to keep the pipeline.yaml for `rote emit`)")
+    return 0
 
 
 def _resolve_eval_skill_dir(
@@ -412,6 +605,7 @@ def _build_scorecard_for(
     pipeline_yaml: Path,
     skill_dir: Path | None,
     provider: str,
+    priors: Priors | None = None,
 ) -> Scorecard:
     """Shared eval flow: estimate both sides, fetch live prices, assemble.
 
@@ -421,13 +615,19 @@ def _build_scorecard_for(
     """
     from datetime import UTC, datetime
 
-    from rote.eval import Priors, estimate_pipeline, estimate_skill, load_eval_estimates
+    from rote.eval import (
+        Priors,
+        estimate_pipeline,
+        estimate_skill,
+        external_call_payload_tokens,
+        load_eval_estimates,
+    )
     from rote.eval.pricing import fetch_catalog
     from rote.eval.scorecard import build_scorecard
     from rote.eval.sidecar import EVAL_SIDECAR_FILENAME
     from rote.eval.tokens import pick_token_counter
 
-    priors = Priors()
+    priors = priors or Priors()
     prices = fetch_catalog(provider=provider).sample(provider=provider)
     # Exact counting needs a live model id (tokenizers are per-model);
     # the small tier's is as good as any and free either way. Only the
@@ -440,7 +640,12 @@ def _build_scorecard_for(
     if skill_dir is not None:
         sidecar_path = pipeline_yaml.parent / EVAL_SIDECAR_FILENAME
         sidecar = load_eval_estimates(sidecar_path) if sidecar_path.is_file() else None
-        skill_estimate = estimate_skill(skill_dir, counter, priors, sidecar=sidecar)
+        # The agent pulls the same sources the pipeline's external_call nodes
+        # bind to; use that footprint to size the before-side context payload.
+        data_payload = external_call_payload_tokens(pipeline, priors)
+        skill_estimate = estimate_skill(
+            skill_dir, counter, priors, sidecar=sidecar, data_payload_tokens=data_payload
+        )
 
     return build_scorecard(
         pipeline_name=pipeline.name,
@@ -490,8 +695,18 @@ def _cmd_eval(args: argparse.Namespace) -> int:
             file=sys.stderr,
         )
 
+    from rote.eval.priors import priors_from_overrides
+
     try:
-        scorecard = _build_scorecard_for(pipeline, pipeline_yaml, skill_dir, args.provider)
+        priors = priors_from_overrides(args.prior or [])
+    except ValueError as e:
+        print(f"error: {e}", file=sys.stderr)
+        return 2
+
+    try:
+        scorecard = _build_scorecard_for(
+            pipeline, pipeline_yaml, skill_dir, args.provider, priors=priors
+        )
     except PricingError as e:
         print(f"error: could not fetch live prices: {e}", file=sys.stderr)
         return 1
@@ -942,12 +1157,48 @@ def _build_parser() -> argparse.ArgumentParser:
     )
     serve.set_defaults(func=_cmd_serve)
 
-    # rote analyze (stub)
+    # rote analyze
     analyze = subparsers.add_parser(
         "analyze",
-        help="Run the graduator against a skill and print a report only",
+        help="Dry-run the graduator: report a skill's graduated shape, emit no runtime code",
+        description=(
+            "Run the rote-graduate agent against a skill and print what "
+            "graduation would produce — node-kind breakdown, roteness "
+            "(deterministic vs. LLM-sampled steps), mandatory checks, HITL "
+            "gates, agent loops, and which runtimes can target it — without "
+            "emitting any runtime code. This is the `plan` to `graduate`'s "
+            "`apply`. Pass --out to keep the pipeline.yaml for a later `rote emit`."
+        ),
+    )
+    analyze.add_argument("skill_path", help="Path to the source skill directory")
+    analyze.add_argument(
+        "--agent",
+        choices=["claude", "codex", "api"],
+        default=None,
+        help=(
+            "Agent runtime for the graduator (default: auto-detect — claude "
+            "CLI first, then codex, then the anthropic API)."
+        ),
+    )
+    analyze.add_argument(
+        "--model",
+        default=None,
+        help="Override the LLM model the graduator uses (default: the driver's default).",
+    )
+    analyze.add_argument(
+        "--out",
+        default=None,
+        help=(
+            "Keep the graduated IR + stubs in this directory (ready for "
+            "`rote emit`). Default: produce them in a temp dir and discard "
+            "after reporting."
+        ),
+    )
+    analyze.add_argument(
+        "--json",
+        action="store_true",
+        help="Emit the analysis report as JSON instead of text.",
     )
-    analyze.add_argument("skill_path")
     analyze.set_defaults(func=_cmd_analyze)
 
     # rote eval
@@ -978,6 +1229,18 @@ def _build_parser() -> argparse.ArgumentParser:
         default="anthropic",
         help="LLM provider whose current model lineup to price (default: anthropic)",
     )
+    eval_cmd.add_argument(
+        "--prior",
+        action="append",
+        default=None,
+        metavar="KEY=VALUE",
+        help=(
+            "Override an estimator prior (repeatable), e.g. "
+            "--prior transcript_growth_per_turn=5962 — feed back the "
+            "re-fits a previous --run reported. Per-MCP-tool payload: "
+            "--prior payload_tokens_per_tool.<tool>=12000"
+        ),
+    )
     eval_cmd.add_argument(
         "--json",
         action="store_true",

{rote_cli-0.6.0 → rote_cli-0.7.0}/src/rote/eval/__init__.py

@@ -25,8 +25,9 @@ from rote.eval.estimate import (
     SamplingSurface,
     estimate_pipeline,
     estimate_skill,
+    external_call_payload_tokens,
 )
-from rote.eval.priors import Priors
+from rote.eval.priors import Priors, priors_from_overrides
 from rote.eval.sidecar import EvalEstimates, load_eval_estimates
 
 __all__ = [
@@ -39,5 +40,7 @@ __all__ = [
     "SamplingSurface",
     "estimate_pipeline",
     "estimate_skill",
+    "external_call_payload_tokens",
     "load_eval_estimates",
+    "priors_from_overrides",
 ]

{rote_cli-0.6.0 → rote_cli-0.7.0}/src/rote/eval/empirical.py

@@ -38,6 +38,7 @@ from typing import Any
 import yaml
 
 from rote.eval.pricing import PricingCatalog
+from rote.eval.priors import Priors
 from rote.graduator.drivers.claude import (
     DEFAULT_ALLOWED_TOOLS,
     build_subscription_env,
@@ -447,9 +448,12 @@ def measured_pipeline_cost_usd(
     return (mean(per_run) if per_run else 0.0, sorted(unpriced))
 
 
-def suggested_priors(skill_runs: tuple[MeasuredRun, ...]) -> dict[str, float]:
+def suggested_priors(
+    skill_runs: tuple[MeasuredRun, ...], priors: Priors | None = None
+) -> dict[str, float]:
     """Prior values re-fitted from measured agent runs (reported, never
     silently applied — the static model's constants stay explicit)."""
+    priors = priors or Priors()
     fitted: dict[str, float] = {}
     timed = [r for r in skill_runs if r.turns and r.wall_seconds]
     if timed:
@@ -462,6 +466,19 @@ def suggested_priors(skill_runs: tuple[MeasuredRun, ...]) -> dict[str, float]:
             mean(r.output_tokens / r.turns for r in with_output if r.turns and r.output_tokens),
             1,
         )
+    # Effective transcript growth Δ, inverted from the measured cache-read
+    # under the quadratic model: cached ≈ (N−1)·C₀ + Δ·(N−2)(N−1)/2, with
+    # C₀ = system_overhead prior. This is *payload-inclusive* — a value far
+    # above the text-only default flags a data-heavy skill whose
+    # tokens_per_external_call_result should be raised. Needs N ≥ 3.
+    c0 = float(priors.system_overhead_tokens)
+    deltas = [
+        max(0.0, 2.0 * (r.cache_read_tokens - (r.turns - 1) * c0) / ((r.turns - 2) * (r.turns - 1)))
+        for r in skill_runs
+        if r.turns and r.turns >= 3 and r.cache_read_tokens
+    ]
+    if deltas:
+        fitted["transcript_growth_per_turn"] = round(mean(deltas), 1)
     return fitted
 
 

{rote_cli-0.6.0 → rote_cli-0.7.0}/src/rote/eval/estimate.py

@@ -308,6 +308,32 @@ def estimate_pipeline(
 
 # ───────── Before: raw-skill estimator ─────────
 
+
+def external_call_payload_tokens(pipeline: Pipeline, priors: Priors | None = None) -> float:
+    """Estimate the data the agent pulls into context, from the pipeline's
+    ``external_call`` footprint.
+
+    The graduated pipeline's ``external_call`` nodes name the sources the
+    skill fetches (Slack, Gmail, a Drive file …); the *before*-side agent
+    pulls the same data, so its transcript carries the same payload. Each
+    external_call contributes ``payload_tokens_per_tool[tool]`` when its MCP
+    tool is known and pinned, otherwise ``tokens_per_external_call_result``.
+
+    Uses the same wave decomposition as :func:`estimate_pipeline`, so
+    loop-body sub-nodes are not double-counted.
+    """
+    priors = priors or Priors()
+    total = 0.0
+    for wave in _execution_waves(pipeline):
+        for node in wave:
+            if node.kind is not NodeKind.EXTERNAL_CALL:
+                continue
+            tool = node.mcp.tool if node.mcp is not None else None
+            per_tool = priors.payload_tokens_per_tool.get(tool) if tool else None
+            total += per_tool if per_tool is not None else priors.tokens_per_external_call_result
+    return total
+
+
 _STEP_LINE_RE = re.compile(r"^\s{0,3}(?:\d+[.)]\s|#{2,3}\s|[-*]\s+\*\*)", re.MULTILINE)
 
 
@@ -339,6 +365,7 @@ def estimate_skill(
     counter: TokenCounter,
     priors: Priors | None = None,
     sidecar: EvalEstimates | None = None,
+    data_payload_tokens: float = 0.0,
 ) -> AgentRunEstimate:
     """Predict running the skill as raw agent instructions.
 
@@ -348,11 +375,18 @@ def estimate_skill(
 
     * fresh  ≈ C₀ + (N−1)·Δ
     * cached ≈ (N−1)·C₀ + Δ·(N−2)(N−1)/2
+
+    ``data_payload_tokens`` (the sources the skill pulls, from
+    :func:`external_call_payload_tokens`) folds into C₀: fetched-once data
+    is billed as one cache-write and re-read on every later turn — exactly
+    C₀'s billing shape, and the dominant cost on data-heavy skills.
     """
     priors = priors or Priors()
     skill_path = Path(skill_dir)
     corpus = _skill_corpus(skill_path)
-    context_tokens = counter.count(corpus) + priors.system_overhead_tokens
+    context_tokens = (
+        counter.count(corpus) + priors.system_overhead_tokens + round(data_payload_tokens)
+    )
 
     if sidecar is not None and (sidecar.steps or sidecar.totals is not None):
         tr = sidecar.turn_range()