runapprentice 0.2.0__tar.gz → 0.3.0__tar.gz

{runapprentice-0.2.0 → runapprentice-0.3.0}/.gitignore

@@ -34,5 +34,21 @@ optimized_program.json
 apprentice-ui/codex_working.md
 .codex/
 
-# Local dev databases
+# Local dev databases (and their manual backups)
 *.db
+*.db.bak*
+
+# Local machine config with secrets — never commit
+.mcp.json
+
+# Local working artifacts, not repo content
+# (.claude/skills holds symlinks into the ignored .agents/ dir — dangling on a fresh clone)
+.claude/skills/
+outputs/
+test-sdk/
+.agents/
+skills-lock.json
+docs/personal_learning.md
+.uv-cache/
+spike/Capture-*.png
+apprentice-ui/ClaudeDesign/screenshots/localhost-*.png

{runapprentice-0.2.0 → runapprentice-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: runapprentice
-Version: 0.2.0
+Version: 0.3.0
 Summary: Cut LLM cost without losing quality: capture traffic, build golden datasets, and optimize prompts.
 Project-URL: Homepage, https://runapprentice.com
 Project-URL: Documentation, https://docs.runapprentice.com
@@ -230,9 +230,23 @@ count and the minimum).
 | `client.datasets.status(task)` | `DatasetStatus(gold, silver, raw, ready_for_optimization)` | same |
 | `client.prompts.register(task, template)` | dict | `ImportError` if a LangChain template is passed without the `[langchain]` extra |
 | `client.prompts.to_langchain(task= or artifact=)` | LangChain prompt | `ImportError` without the `[langchain]` extra |
-| `client.optimize(task)` | `Job` | 400 if fewer than the backend's configured verified-row minimum (20 by default) |
+| `client.optimize(task, baseline_prompt=None, budget=None, metric=None)` | `Job` | 400 if fewer than the backend's configured verified-row minimum (20 by default) |
+| `client.preview_optimization(task, baseline_prompt=None, budget=None, metric=None)` | `OptimizationPreview` | `ApprenticeError` on API failure |
+| `client.train(task)` | `Job` | 400 if fewer gold rows than the backend's training minimum |
 | `client.job(job_id)` / `job.refresh()` / `job.wait(poll_seconds=, timeout_seconds=)` | `Job` | `ApprenticeError` on timeout |
-| `job.report()` | `OptimizationReport(baseline_score, optimized_score, optimized_prompt, ...)` | if job has no report |
+| `client.cancel_job(job_id)` | dict (`cancel_requested`, `status`) | 409 if job is not cancellable |
+| `job.cancel()` | dict (`cancel_requested`, `status`) | 409 if job is not cancellable |
+| `client.jobs(task)` | `list[JobSummary]` | `ApprenticeError` on API failure |
+| `client.local_runs(task)` | `list[LocalRun]` | `ApprenticeError` on API failure |
+| `client.datasets.rows(task, tier, limit=200)` | `list[DatasetRow]` | `ApprenticeError` on API failure |
+| `client.optimize_local(task, data=None, **options)` | dict (local GEPA result) | `ApprenticeError` if no `data` and no API key |
+| `client.train_local(task, data=None, effort=None, **options)` | dict (local MLX result) | `ApprenticeError` if no `data` and no API key |
+| `client.register_local_run(task, base_model, dataset_row_count, seed, config=None)` | `LocalRun` | `ApprenticeError` on API failure |
+| `client.complete_local_run(run_id, payload)` | dict | `ApprenticeError` on API failure |
+| `client.capture(task, output, input= or inputs=, model=, latency_ms=, ...)` | trace_id or None | **never raises** |
+| `client.flush_captures(timeout=2.0)` | None (blocks until buffered traces deliver or timeout; no-op in sync mode) | nothing by design |
+| `client.close()` | None (flushes and closes transport) | nothing by design |
+| `job.report()` | typed optimize, train, or generate report; raw dict for future shapes | if job has no report |
 | `client.prompts.get(task, version=None)` | `PromptVersion(version, text, score)` | 404 if never optimized |
 | `client.prompts.history(task)` | `list[PromptVersion]` | `ApprenticeError` on API failure |
 | `client.feedback(trace_id, good=, score=, note=)` | None | 404 unknown trace; `ApprenticeError` if pending captures cannot drain in time, or on API failure |

{runapprentice-0.2.0 → runapprentice-0.3.0}/README.md

@@ -201,9 +201,23 @@ count and the minimum).
 | `client.datasets.status(task)` | `DatasetStatus(gold, silver, raw, ready_for_optimization)` | same |
 | `client.prompts.register(task, template)` | dict | `ImportError` if a LangChain template is passed without the `[langchain]` extra |
 | `client.prompts.to_langchain(task= or artifact=)` | LangChain prompt | `ImportError` without the `[langchain]` extra |
-| `client.optimize(task)` | `Job` | 400 if fewer than the backend's configured verified-row minimum (20 by default) |
+| `client.optimize(task, baseline_prompt=None, budget=None, metric=None)` | `Job` | 400 if fewer than the backend's configured verified-row minimum (20 by default) |
+| `client.preview_optimization(task, baseline_prompt=None, budget=None, metric=None)` | `OptimizationPreview` | `ApprenticeError` on API failure |
+| `client.train(task)` | `Job` | 400 if fewer gold rows than the backend's training minimum |
 | `client.job(job_id)` / `job.refresh()` / `job.wait(poll_seconds=, timeout_seconds=)` | `Job` | `ApprenticeError` on timeout |
-| `job.report()` | `OptimizationReport(baseline_score, optimized_score, optimized_prompt, ...)` | if job has no report |
+| `client.cancel_job(job_id)` | dict (`cancel_requested`, `status`) | 409 if job is not cancellable |
+| `job.cancel()` | dict (`cancel_requested`, `status`) | 409 if job is not cancellable |
+| `client.jobs(task)` | `list[JobSummary]` | `ApprenticeError` on API failure |
+| `client.local_runs(task)` | `list[LocalRun]` | `ApprenticeError` on API failure |
+| `client.datasets.rows(task, tier, limit=200)` | `list[DatasetRow]` | `ApprenticeError` on API failure |
+| `client.optimize_local(task, data=None, **options)` | dict (local GEPA result) | `ApprenticeError` if no `data` and no API key |
+| `client.train_local(task, data=None, effort=None, **options)` | dict (local MLX result) | `ApprenticeError` if no `data` and no API key |
+| `client.register_local_run(task, base_model, dataset_row_count, seed, config=None)` | `LocalRun` | `ApprenticeError` on API failure |
+| `client.complete_local_run(run_id, payload)` | dict | `ApprenticeError` on API failure |
+| `client.capture(task, output, input= or inputs=, model=, latency_ms=, ...)` | trace_id or None | **never raises** |
+| `client.flush_captures(timeout=2.0)` | None (blocks until buffered traces deliver or timeout; no-op in sync mode) | nothing by design |
+| `client.close()` | None (flushes and closes transport) | nothing by design |
+| `job.report()` | typed optimize, train, or generate report; raw dict for future shapes | if job has no report |
 | `client.prompts.get(task, version=None)` | `PromptVersion(version, text, score)` | 404 if never optimized |
 | `client.prompts.history(task)` | `list[PromptVersion]` | `ApprenticeError` on API failure |
 | `client.feedback(trace_id, good=, score=, note=)` | None | 404 unknown trace; `ApprenticeError` if pending captures cannot drain in time, or on API failure |

{runapprentice-0.2.0 → runapprentice-0.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "runapprentice"
-version = "0.2.0"
+version = "0.3.0"
 description = "Cut LLM cost without losing quality: capture traffic, build golden datasets, and optimize prompts."
 readme = "README.md"
 requires-python = ">=3.11"

{runapprentice-0.2.0 → runapprentice-0.3.0}/src/runapprentice/__init__.py

@@ -4,12 +4,16 @@ from runapprentice.client import Apprentice, ApprenticeError, Job, enable_debug_
 from runapprentice.models import (
     DatasetRow,
     DatasetStatus,
+    GenerationReport,
     JobInfo,
+    JobReport,
     JobSummary,
     LocalRun,
     OptimizationReport,
+    OptimizationPreview,
     PromptVersion,
     TraceRecord,
+    TrainingReport,
 )
 
 __all__ = [
@@ -19,12 +23,23 @@ __all__ = [
     "Job",
     "DatasetRow",
     "DatasetStatus",
+    "GenerationReport",
     "JobInfo",
+    "JobReport",
     "JobSummary",
     "LocalRun",
     "OptimizationReport",
+    "OptimizationPreview",
     "PromptVersion",
     "TraceRecord",
+    "TrainingReport",
 ]
 
-__version__ = "0.1.0"
+# Single source of truth is pyproject.toml; read it from package metadata so
+# this string can never drift from what PyPI serves (it did once, at 0.2.0).
+from importlib.metadata import PackageNotFoundError, version as _version
+
+try:
+    __version__ = _version("runapprentice")
+except PackageNotFoundError:  # running from a source tree without installation
+    __version__ = "0.0.0.dev0"

{runapprentice-0.2.0 → runapprentice-0.3.0}/src/runapprentice/cli.py

@@ -13,7 +13,13 @@ from typing import Any
 
 from runapprentice.client import Apprentice, ApprenticeError
 from runapprentice import local as _local
-from runapprentice.models import JobInfo, JobSummary, OptimizationReport
+from runapprentice.models import (
+    JobInfo,
+    JobReport,
+    JobSummary,
+    OptimizationPreview,
+    OptimizationReport,
+)
 
 CONFIG_DIR = Path("~/.apprentice").expanduser()
 CONFIG_PATH = CONFIG_DIR / "config.json"
@@ -107,6 +113,22 @@ def build_parser() -> argparse.ArgumentParser:
         type=Path,
         help="CSV with input,output columns; required for --local unless an API key is set",
     )
+    optimize.add_argument(
+        "--budget",
+        choices=("quick", "thorough"),
+        help="hosted optimization budget; omit for balanced default",
+    )
+    optimize.add_argument("--metric", help="metric used to score this hosted run")
+    optimize.add_argument(
+        "--prompt-file",
+        type=Path,
+        help="read hosted run baseline prompt from this UTF-8 text file",
+    )
+    optimize.add_argument(
+        "--preview",
+        action="store_true",
+        help="show hosted metric and cost estimate without enqueueing a run",
+    )
     optimize.set_defaults(func=_optimize)
 
     train = sub.add_parser(
@@ -153,6 +175,10 @@ def build_parser() -> argparse.ArgumentParser:
     job = sub.add_parser("job", help="details for one hosted job")
     job.add_argument("job_id", help="job id from `apprentice jobs`")
     job.set_defaults(func=_job)
+
+    cancel = sub.add_parser("cancel", help="cancel a queued or running hosted job")
+    cancel.add_argument("job_id", help="job id from `apprentice jobs`")
+    cancel.set_defaults(func=_cancel)
     return parser
 
 
@@ -180,6 +206,11 @@ def _status(args: argparse.Namespace, client: Apprentice) -> int:
 
 def _optimize(args: argparse.Namespace, client: Apprentice) -> int:
     if args.local:
+        if args.budget or args.metric or args.prompt_file or args.preview:
+            return _fail(
+                "--budget, --metric, --prompt-file, and --preview apply to hosted optimization",
+                bool(args.json),
+            )
         return _optimize_local(
             args.task,
             args.data,
@@ -187,7 +218,30 @@ def _optimize(args: argparse.Namespace, client: Apprentice) -> int:
             bool(args.json),
             bool(args._api_key_present),
         )
-    job = client.optimize(args.task).wait()
+    baseline_prompt = None
+    if args.prompt_file is not None:
+        try:
+            baseline_prompt = args.prompt_file.read_text(encoding="utf-8")
+        except OSError as exc:
+            raise ApprenticeError(f"could not read prompt file {args.prompt_file}: {exc}") from exc
+    if args.preview:
+        preview = client.preview_optimization(
+            args.task,
+            baseline_prompt=baseline_prompt,
+            budget=args.budget,
+            metric=args.metric,
+        )
+        return _emit(
+            preview.model_dump(mode="json"),
+            _preview_line(preview),
+            bool(args.json),
+        )
+    job = client.optimize(
+        args.task,
+        baseline_prompt=baseline_prompt,
+        budget=args.budget,
+        metric=args.metric,
+    ).wait()
     payload = job.info.model_dump(mode="json")
     return _emit(payload, _report_line(job.info.report), bool(args.json))
 
@@ -222,6 +276,11 @@ def _job(args: argparse.Namespace, client: Apprentice) -> int:
     return _emit(job.info.model_dump(mode="json"), _job_info_line(job.info), bool(args.json))
 
 
+def _cancel(args: argparse.Namespace, client: Apprentice) -> int:
+    result = client.cancel_job(args.job_id)
+    return _emit(result, f"cancel requested: {args.job_id} ({result['status']})", bool(args.json))
+
+
 def _train_local(
     task: str,
     client: Apprentice,
@@ -424,12 +483,23 @@ def _job_info_line(job: JobInfo) -> str:
     return f"{job.job_id} {job.kind} {job.status}"
 
 
-def _report_line(report: OptimizationReport | None) -> str:
+def _report_line(report: JobReport | None) -> str:
     if report is None:
         return "optimize finished with no report"
+    if not isinstance(report, OptimizationReport):
+        return "optimize finished with an unrecognized report"
     return f"optimize done: baseline={report.baseline_score} optimized={report.optimized_score}"
 
 
+def _preview_line(preview: OptimizationPreview) -> str:
+    estimate = preview.estimate
+    return (
+        f"optimize preview: metric={preview.eval_metric} budget={estimate.get('budget')} "
+        f"rows={estimate.get('rows')} cost_usd={estimate.get('cost_usd_low')}"
+        f"-{estimate.get('cost_usd_high')}"
+    )
+
+
 def _emit(payload: dict[str, Any], text: str, as_json: bool) -> int:
     print(json.dumps(payload, sort_keys=True) if as_json else text)
     return 0

{runapprentice-0.2.0 → runapprentice-0.3.0}/src/runapprentice/client.py

@@ -16,7 +16,7 @@ import os
 import time
 import uuid
 from pathlib import Path
-from typing import Any
+from typing import Any, Literal
 
 import httpx
 
@@ -25,9 +25,10 @@ from runapprentice.models import (
     DatasetRow,
     DatasetStatus,
     JobInfo,
+    JobReport,
     JobSummary,
     LocalRun,
-    OptimizationReport,
+    OptimizationPreview,
     PromptVersion,
     TraceRecord,
 )
@@ -114,10 +115,50 @@ class Apprentice:
             raise ApprenticeError(f"{method} {path} -> {resp.status_code}: {resp.text[:500]}{suffix}")
         return resp.json()
 
-    def optimize(self, task: str) -> Job:
-        data = self._request("POST", "/v1/optimize", json={"task": task})
+    def optimize(
+        self,
+        task: str,
+        baseline_prompt: str | None = None,
+        budget: Literal["quick", "thorough"] | None = None,
+        metric: str | None = None,
+    ) -> Job:
+        payload = {"task": task}
+        payload.update(
+            {
+                key: value
+                for key, value in {
+                    "baseline_prompt": baseline_prompt,
+                    "budget": budget,
+                    "metric": metric,
+                }.items()
+                if value is not None
+            }
+        )
+        data = self._request("POST", "/v1/optimize", json=payload)
         return Job(self, JobInfo.model_validate(data))
 
+    def preview_optimization(
+        self,
+        task: str,
+        baseline_prompt: str | None = None,
+        budget: Literal["quick", "thorough"] | None = None,
+        metric: str | None = None,
+    ) -> OptimizationPreview:
+        payload = {"task": task}
+        payload.update(
+            {
+                key: value
+                for key, value in {
+                    "baseline_prompt": baseline_prompt,
+                    "budget": budget,
+                    "metric": metric,
+                }.items()
+                if value is not None
+            }
+        )
+        data = self._request("POST", "/v1/optimize/preview", json=payload)
+        return OptimizationPreview.model_validate(data)
+
     def job(self, job_id: str) -> Job:
         data = self._request("GET", f"/v1/jobs/{job_id}")
         return Job(self, JobInfo.model_validate(data))
@@ -126,6 +167,13 @@ class Apprentice:
         data = self._request("GET", f"/v1/tasks/{task}/jobs")
         return [JobSummary.model_validate(item) for item in data]
 
+    def cancel_job(self, job_id: str) -> dict[str, Any]:
+        return dict(self._request("POST", f"/v1/jobs/{job_id}/cancel"))
+
+    def local_runs(self, task: str) -> list[LocalRun]:
+        data = self._request("GET", f"/v1/tasks/{task}/local-runs")
+        return [LocalRun.model_validate(item) for item in data]
+
     def train(self, task: str) -> Job:
         data = self._request("POST", f"/v1/tasks/{task}/train", json={})
         return Job(self, JobInfo.model_validate(data))
@@ -350,7 +398,10 @@ class Job:
             time.sleep(poll_seconds)
         return self
 
-    def report(self) -> OptimizationReport:
+    def cancel(self) -> dict[str, Any]:
+        return self._client.cancel_job(self.info.job_id)
+
+    def report(self) -> JobReport:
         if self.info.report is None:
             raise ApprenticeError(
                 f"job {self.info.job_id} has no report (status={self.info.status})"

runapprentice-0.3.0/src/runapprentice/models.py

@@ -0,0 +1,179 @@
+"""Pydantic models for the Apprentice SDK. Mirrors the control-plane API contract."""
+
+from __future__ import annotations
+
+from datetime import datetime
+from typing import Any
+
+from pydantic import BaseModel, Field, SkipValidation, ValidationError, model_validator
+
+
+class DatasetRow(BaseModel):
+    id: str | None = None
+    input: str
+    inputs: dict[str, str] | None = None
+    output: str
+    tier: str | None = None
+    source: str | None = None
+    flag_label: str | None = None
+    feedback_score: float | None = None
+    expert_authored: bool | None = None
+    created_at: datetime | None = None
+    metadata: dict[str, Any] = Field(default_factory=dict)
+
+
+class DatasetStatus(BaseModel):
+    task: str
+    gold: int = 0
+    silver: int = 0
+    raw: int = 0
+    ready_for_optimization: bool = False
+
+
+class TraceRecord(BaseModel):
+    trace_id: str | None = Field(default=None, pattern=r"^[0-9a-f]{32}$")
+    task: str
+    input: str
+    inputs: dict[str, str] | None = None
+    output: str
+    model: str | None = None
+    latency_ms: float | None = None
+    prompt_tokens: int | None = None
+    completion_tokens: int | None = None
+    error: str | None = None
+    metadata: dict[str, Any] = Field(default_factory=dict)
+
+
+class OptimizationReport(BaseModel):
+    task: str
+    baseline_score: float | None = None
+    optimized_score: float | None = None
+    examples_used: int | None = None
+    optimized_prompt: str | None = None
+    optimized_template: dict[str, Any] | None = None
+    detail: dict[str, Any] = Field(default_factory=dict)
+
+
+class TrainingReport(BaseModel):
+    task: str | None = None
+    model_id: str | None = None
+    base_model: str | None = None
+    adapter_ref: str | None = None
+    champion_score: float | None = None
+    challenger_score: float | None = None
+    eval_metric: str | None = None
+    metric_tier: str | None = None
+    holdout_size: int | None = None
+    status: str | None = None
+    reason: str | None = None
+    config: dict[str, Any] | None = None
+
+
+class GenerationReport(BaseModel):
+    rows_inserted: int | None = None
+    rows_rejected: int | None = None
+    reject_reasons: dict[str, int] | None = None
+    model: str | None = None
+    llm_fallback_used: bool | None = None
+    input_tokens: int | None = None
+    output_tokens: int | None = None
+    samples: list[dict[str, Any]] | None = None
+    chunks_truncated: int | None = None
+    response_failures: int | None = None
+    duplicate_inputs: int | None = None
+    executor_requested: str | None = None
+    executor_actual_models: list[str] | None = None
+    executor_fallback_used: bool | None = None
+
+
+JobReport = OptimizationReport | TrainingReport | GenerationReport | dict[str, Any]
+
+
+class JobInfo(BaseModel):
+    job_id: str
+    task: str  # task ID; use task_name when calling task-scoped client methods
+    task_name: str | None = None
+    kind: str
+    status: str
+    created_at: datetime | None = None
+    error: str | None = None
+    report: SkipValidation[JobReport] | None = None
+    progress: dict[str, Any] | None = None
+    queue_position: int | None = None
+    baseline_score: float | None = None
+    optimized_score: float | None = None
+    baseline_from_user: bool | None = None
+    generation_session_id: str | None = None
+    round: int | None = None
+    predecessor_job_id: str | None = None
+    phase: str | None = None
+    guidance: str | None = None
+
+    @model_validator(mode="before")
+    @classmethod
+    def parse_report_for_kind(cls, value: Any) -> Any:
+        if not isinstance(value, dict) or not isinstance(value.get("report"), dict):
+            return value
+        kind = value.get("kind")
+        if kind == "optimize":
+            report_type: type[BaseModel] = OptimizationReport
+        elif kind == "train":
+            report_type = TrainingReport
+        elif kind == "generate":
+            report_type = GenerationReport
+        else:
+            return value
+        try:
+            report = report_type.model_validate(value["report"])
+        except ValidationError:
+            return value
+        return {**value, "report": report}
+
+
+class JobSummary(BaseModel):
+    job_id: str
+    task_name: str | None = None
+    status: str
+    kind: str
+    created_at: datetime | None = None
+    baseline_score: float | None = None
+    optimized_score: float | None = None
+
+
+class LocalRun(BaseModel):
+    run_id: str
+    task: str | None = None
+    task_id: str | None = None
+    status: str = "registered"
+    eval_tier: str | None = None
+    base_model: str | None = None
+    dataset_row_count: int | None = None
+    seed: int | None = None
+    baseline_score: float | None = None
+    final_score: float | None = None
+    held_out_rows: int | None = None
+    adapter_sha256: str | None = None
+    mlx_lm_version: str | None = None
+    error_message: str | None = None
+    created_at: datetime | None = None
+    completed_at: datetime | None = None
+
+
+class OptimizationPreview(BaseModel):
+    eval_metric: str
+    metric_tier: str
+    requested_metric: str
+    recommended_metric: str
+    metric_reason: str
+    rag_candidate: bool
+    rows_checked: int
+    rows_source: str
+    guidance: dict[str, Any]
+    estimate: dict[str, Any]
+
+
+class PromptVersion(BaseModel):
+    task: str
+    version: int
+    text: str
+    score: float | None = None

{runapprentice-0.2.0 → runapprentice-0.3.0}/tests/test_cli.py

@@ -9,7 +9,7 @@ import pytest
 
 from runapprentice import ApprenticeError, cli
 import runapprentice.local as local
-from runapprentice.models import DatasetRow
+from runapprentice.models import DatasetRow, JobInfo, JobSummary
 
 
 def test_parser_supports_required_commands() -> None:
@@ -28,6 +28,44 @@ def test_parser_supports_required_commands() -> None:
     assert awake.fresh is True
     assert parser.parse_args(["jobs", "task"]).command == "jobs"
     assert parser.parse_args(["job", "j1"]).command == "job"
+    controls = parser.parse_args(
+        [
+            "optimize",
+            "task",
+            "--budget",
+            "thorough",
+            "--metric",
+            "json_f1",
+            "--prompt-file",
+            "prompt.txt",
+            "--preview",
+        ]
+    )
+    assert controls.budget == "thorough"
+    assert controls.metric == "json_f1"
+    assert controls.prompt_file == Path("prompt.txt")
+    assert controls.preview is True
+    assert parser.parse_args(["cancel", "j1"]).command == "cancel"
+
+
+def test_cli_renders_all_job_kinds() -> None:
+    for kind, report in (
+        ("optimize", {"task": "extract", "baseline_score": 50.0}),
+        ("train", {"task": "extract", "champion_score": 80.0}),
+        ("generate", {"rows_inserted": 10, "rows_rejected": 1}),
+    ):
+        info = JobInfo.model_validate(
+            {
+                "job_id": f"j-{kind}",
+                "task": "task-id",
+                "kind": kind,
+                "status": "succeeded",
+                "report": report,
+            }
+        )
+        summary = JobSummary(job_id=info.job_id, kind=kind, status=info.status)
+        assert cli._job_info_line(info) == f"j-{kind} {kind} succeeded"
+        assert cli._job_summary(summary)["kind"] == kind
 
 
 def test_effort_explicit_overrides_hardware_detection() -> None:

{runapprentice-0.2.0 → runapprentice-0.3.0}/tests/test_client.py

@@ -13,7 +13,11 @@ import pytest
 from runapprentice import Apprentice, ApprenticeError, TraceRecord
 from runapprentice._capture import CaptureDispatcher
 import runapprentice.local as local
-from runapprentice.models import OptimizationReport
+from runapprentice.models import (
+    GenerationReport,
+    OptimizationReport,
+    TrainingReport,
+)
 
 
 def make_client(routes: dict[str, Any]) -> tuple[Apprentice, list[dict[str, Any]]]:
@@ -219,6 +223,179 @@ def test_optimize_and_job_lifecycle() -> None:
     assert report.optimized_score == 88.0
 
 
+def test_generate_job_payload_parses_without_crashing() -> None:
+    payload = {
+        "job_id": "j-gen",
+        "task": "task-id",
+        "task_name": "invoices",
+        "kind": "generate",
+        "status": "succeeded",
+        "generation_session_id": "session-1",
+        "round": 2,
+        "predecessor_job_id": "j-preview",
+        "phase": "full",
+        "guidance": "Include boundary cases",
+        "report": {
+            "rows_inserted": 17,
+            "rows_rejected": 3,
+            "reject_reasons": {"duplicate": 2, "invalid_output": 1},
+            "model": "generator-model",
+            "llm_fallback_used": False,
+            "input_tokens": 1200,
+            "output_tokens": 700,
+            "samples": [{"input": "sample", "output": "{}"}],
+            "chunks_truncated": 1,
+            "response_failures": 1,
+        },
+    }
+    client, _ = make_client({"GET /v1/jobs/j-gen": payload})
+
+    job = client.job("j-gen")
+
+    report = job.report()
+    assert isinstance(report, GenerationReport)
+    assert report.rows_inserted == 17
+    assert job.info.generation_session_id == "session-1"
+    assert job.info.task_name == "invoices"
+
+
+def test_train_report_keeps_training_scores() -> None:
+    client, _ = make_client(
+        {
+            "GET /v1/jobs/j-train": {
+                "job_id": "j-train",
+                "task": "task-id",
+                "kind": "train",
+                "status": "succeeded",
+                "report": {
+                    "task": "extract",
+                    "model_id": "model-1",
+                    "base_model": "base-model",
+                    "adapter_ref": "adapter-1",
+                    "champion_score": 81.5,
+                    "challenger_score": 84.0,
+                    "eval_metric": "json_f1",
+                    "metric_tier": "deterministic",
+                    "holdout_size": 50,
+                    "status": "ready",
+                    "reason": "challenger met champion and minimum score",
+                    "config": {"seed": 42},
+                },
+            }
+        }
+    )
+
+    report = client.job("j-train").report()
+
+    assert isinstance(report, TrainingReport)
+    assert report.model_dump()["champion_score"] == 81.5
+    assert report.model_dump()["challenger_score"] == 84.0
+
+
+def test_unknown_or_malformed_report_stays_raw() -> None:
+    client, _ = make_client(
+        {
+            "GET /v1/jobs/j-future": {
+                "job_id": "j-future",
+                "task": "task-id",
+                "kind": "future-kind",
+                "status": "succeeded",
+                "report": {"new_field": [1, 2, 3]},
+            },
+            "GET /v1/jobs/j-bad": {
+                "job_id": "j-bad",
+                "task": "task-id",
+                "kind": "optimize",
+                "status": "succeeded",
+                "report": {"baseline_score": "not-a-number"},
+            },
+        }
+    )
+
+    assert client.job("j-future").report() == {"new_field": [1, 2, 3]}
+    assert client.job("j-bad").report() == {"baseline_score": "not-a-number"}
+
+
+def test_optimize_sends_only_set_controls_and_preview_parses() -> None:
+    preview = {
+        "eval_metric": "json_f1",
+        "metric_tier": "deterministic",
+        "requested_metric": "json_f1",
+        "recommended_metric": "json_f1",
+        "metric_reason": "outputs are JSON objects",
+        "rag_candidate": False,
+        "rows_checked": 20,
+        "rows_source": "verified",
+        "guidance": {
+            "use_case": "json_extraction",
+            "recommended_min_rows": 30,
+            "verified_rows": 20,
+            "data_adequacy": "limited",
+            "best_practice": "Add more verified rows.",
+        },
+        "estimate": {
+            "rows": 20,
+            "budget": "heavy",
+            "cost_usd_low": 0.2,
+            "cost_usd_high": 0.5,
+        },
+    }
+    client, received = make_client(
+        {
+            "POST /v1/optimize": {
+                "job_id": "j1",
+                "task": "task-id",
+                "kind": "optimize",
+                "status": "queued",
+            },
+            "POST /v1/optimize/preview": preview,
+        }
+    )
+
+    client.optimize("extract", budget="thorough")
+    result = client.preview_optimization("extract", metric="json_f1")
+
+    assert received[0]["body"] == {"task": "extract", "budget": "thorough"}
+    assert received[1]["body"] == {"task": "extract", "metric": "json_f1"}
+    assert result.estimate["cost_usd_high"] == 0.5
+
+
+def test_cancel_happy_path_and_job_cancel() -> None:
+    client, received = make_client(
+        {
+            "GET /v1/jobs/j1": {
+                "job_id": "j1",
+                "task": "task-id",
+                "kind": "optimize",
+                "status": "running",
+            },
+            "POST /v1/jobs/j1/cancel": {
+                "job_id": "j1",
+                "cancel_requested": True,
+                "status": "running",
+            },
+        }
+    )
+
+    assert client.job("j1").cancel()["cancel_requested"] is True
+    assert received[-1] == {"method": "POST", "path": "/v1/jobs/j1/cancel", "body": None}
+
+
+def test_cancel_409_uses_normal_error_path() -> None:
+    client = Apprentice(api_key="test", base_url="http://mock", capture_async=False)
+    client._http = httpx.Client(
+        base_url="http://mock",
+        transport=httpx.MockTransport(
+            lambda _request: httpx.Response(
+                409, json={"detail": "job j1 is succeeded, nothing to cancel"}
+            )
+        ),
+    )
+
+    with pytest.raises(ApprenticeError, match="nothing to cancel"):
+        client.cancel_job("j1")
+
+
 def test_train_jobs_rows_and_local_run_methods() -> None:
     client, received = make_client(
         {
@@ -232,20 +409,49 @@ def test_train_jobs_rows_and_local_run_methods() -> None:
                 {"job_id": "j1", "kind": "optimize", "status": "succeeded"}
             ],
             "GET /v1/datasets/t1/rows": {
-                "rows": [{"id": "r1", "input": "i", "output": "o", "tier": "gold"}],
+                "rows": [
+                    {
+                        "id": "r1",
+                        "input": "i",
+                        "output": "o",
+                        "tier": "gold",
+                        "source": "review",
+                        "flag_label": "edge_case",
+                        "feedback_score": 1.0,
+                        "expert_authored": True,
+                        "created_at": "2026-07-18T00:00:00+00:00",
+                    }
+                ],
                 "total": 1,
             },
             "POST /v1/local-runs": {"run_id": "lr1", "task": "t1", "status": "registered"},
             "POST /v1/local-runs/lr1/complete": {"ok": True},
+            "GET /v1/tasks/t1/local-runs": [
+                {
+                    "run_id": "lr1",
+                    "task_id": "task-id",
+                    "status": "completed",
+                    "eval_tier": "self_reported",
+                    "base_model": "m",
+                    "dataset_row_count": 3,
+                    "seed": 42,
+                    "final_score": 91.0,
+                }
+            ],
         }
     )
 
     assert client.train("t1").job_id == "j2"
     assert client.jobs("t1")[0].job_id == "j1"
-    assert client.datasets.rows("t1", "gold")[0].id == "r1"
+    row = client.datasets.rows("t1", "gold")[0]
+    assert row.id == "r1"
+    assert row.source == "review"
+    assert row.expert_authored is True
     assert client.register_local_run("t1", "m", 3, 42).run_id == "lr1"
     assert client.complete_local_run("lr1", {"final_score": 1.0}) == {"ok": True}
-    assert received[-2]["body"] == {
+    assert client.local_runs("t1")[0].final_score == 91.0
+    registration = next(item for item in received if item["path"] == "/v1/local-runs")
+    assert registration["body"] == {
         "task": "t1",
         "base_model": "m",
         "dataset_row_count": 3,

runapprentice-0.2.0/src/runapprentice/models.py

@@ -1,81 +0,0 @@
-"""Pydantic models for the Apprentice SDK. Mirrors the control-plane API contract."""
-
-from __future__ import annotations
-
-from datetime import datetime
-from typing import Any
-
-from pydantic import BaseModel, Field
-
-
-class DatasetRow(BaseModel):
-    id: str | None = None
-    input: str
-    inputs: dict[str, str] | None = None
-    output: str
-    tier: str | None = None
-    metadata: dict[str, Any] = Field(default_factory=dict)
-
-
-class DatasetStatus(BaseModel):
-    task: str
-    gold: int = 0
-    silver: int = 0
-    raw: int = 0
-    ready_for_optimization: bool = False
-
-
-class TraceRecord(BaseModel):
-    trace_id: str | None = Field(default=None, pattern=r"^[0-9a-f]{32}$")
-    task: str
-    input: str
-    inputs: dict[str, str] | None = None
-    output: str
-    model: str | None = None
-    latency_ms: float | None = None
-    prompt_tokens: int | None = None
-    completion_tokens: int | None = None
-    error: str | None = None
-    metadata: dict[str, Any] = Field(default_factory=dict)
-
-
-class OptimizationReport(BaseModel):
-    task: str
-    baseline_score: float | None = None
-    optimized_score: float | None = None
-    examples_used: int | None = None
-    optimized_prompt: str | None = None
-    optimized_template: dict[str, Any] | None = None
-    detail: dict[str, Any] = Field(default_factory=dict)
-
-
-class JobInfo(BaseModel):
-    job_id: str
-    task: str
-    kind: str
-    status: str  # queued | running | succeeded | failed
-    created_at: datetime | None = None
-    error: str | None = None
-    report: OptimizationReport | None = None
-
-
-class JobSummary(BaseModel):
-    job_id: str
-    status: str
-    kind: str
-    created_at: datetime | None = None
-    baseline_score: float | None = None
-    optimized_score: float | None = None
-
-
-class LocalRun(BaseModel):
-    run_id: str
-    task: str
-    status: str = "registered"
-
-
-class PromptVersion(BaseModel):
-    task: str
-    version: int
-    text: str
-    score: float | None = None