agentbyte 0.33.0__tar.gz → 0.34.0__tar.gz

{agentbyte-0.33.0 → agentbyte-0.34.0}/.gitignore

@@ -64,3 +64,10 @@ data/flatfiles/*
 
 # lean-ctx context runtime cache
 .lean-ctx/
+
+# Local SQLite stores (evaluation history, checkpoints) and their writer locks
+*.sqlite
+*.sqlite-journal
+*.sqlite-wal
+*.sqlite-shm
+*.writer.lock

{agentbyte-0.33.0 → agentbyte-0.34.0}/CHANGELOG.md

@@ -6,6 +6,23 @@ The format follows Keep a Changelog principles and semantic versioning.
 
 ## [Unreleased]
 
+## [0.34.0] - 2026-10-01
+
+### Added
+
+- Evaluation workbench, step 3 (spec 0052): WebUI Evaluations section next to the Playground — registered suites with Run (summary or full evidence), run history with pass/fail verdicts and filters, a live run page (durable snapshots, outcome counts, Cancel) and an item page (transcript, judge scores and reasoning, execution facts, labelled evidence state). Pages have shareable `#/evals/...` links. A server with evaluations but no agents or workflows opens on Evaluations.
+- `/api/health` reports `eval_enabled`; `GET /api/eval/runs` entries include a `verdict` with pass/fail and outcome counts.
+- Simpler eval commands: `--catalog` / `--eval-catalog` (and `registry check`) accept a `.py` file path as well as a module, and `:name` is optional when the module defines exactly one matching object. `agentbyte eval run|results` and `agentbyte webui --eval-catalog` store runs in `./eval.sqlite` unless a history path is given. Example: `agentbyte webui --eval-catalog examples/eval/workbench_catalog.py`.
+- `agentbyte eval ui [--catalog CATALOG] [--history PATH] [--port 8090]`: the Evaluations pages on their own, without the Playground (`AgentbyteWebUIServer(playground=False)`, `serve`/`webui(playground=False)`); `/api/health` reports `playground_enabled`. It refuses to start, with exit code 2, when another process holds the history file.
+- `agentbyte.eval.tasks_from_eval_records(records)`: the tasks a saved eval dataset would produce, built synchronously from in-memory `EvalDatasetRecord`s (for suite catalogs). Saving and loading now share one record/row mapping.
+- Evaluation guide `docs/guides/evaluation-end-to-end.md` (how datasets, suites, runners, history, CLI, HTTP and the UI connect; on-the-fly vs stored runs; CI), the real-model example catalog `examples/eval/acme_support.py` with its fake FAQ dataset (`data/datasets/examples/acme-support`), and the notebook `notebooks/usecases/eval/06-eval-end-to-end.ipynb`.
+- The sample WebUI launchers (`examples/webui/in_memory.py`, `presets_webui.py`) accept `--eval-history`, `--eval-catalog`, `--port` and `--no-open`, serving the demo entities and the Evaluations pages together.
+
+### Fixed
+
+- The packaged WebUI called `http://localhost:8080` regardless of the port it was served from; it now calls its own origin, so `agentbyte webui --port N` works.
+- README "Run The WebUI": removed the `--dir examples` suggestion (discovery finds no entities there) and documented the sample launchers, Evaluations and the discovery convention.
+
 ## [0.33.0] - 2026-10-01
 
 Upgrade notes (breaking):

{agentbyte-0.33.0 → agentbyte-0.34.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: agentbyte
-Version: 0.33.0
+Version: 0.34.0
 Summary: A toolkit for designing multiagent systems
 Author-email: MrDataPsycho <mr.data.psycho@gmail.com>
 License-Expression: LicenseRef-Proprietary
@@ -72,7 +72,7 @@ Description-Content-Type: text/markdown
 
 Agentbyte is an observability-first agentic AI framework for building and studying multiagent systems with a learning-first, implementation-oriented workflow.
 
-Current release: **0.33.0**
+Current release: **0.34.0**
 
 ## Building an Agent
 
@@ -323,110 +323,87 @@ uv sync --all-extras --all-groups
 
 ## Run The WebUI
 
-### Option 1: Run the preset-backed app
-
-This is the easiest way to see the WebUI working end to end with real preset entities:
-- preset agents
-- preset orchestrators
-- preset workflow
-
-Step 1. Install the WebUI extra:
-
-```bash
-uv sync --extra webui
-```
-
-Step 2. Start the preset-backed app:
+The WebUI has two sections: the **Playground** (chat with agents, orchestrators and workflows) and
+**Evaluations** (run registered evaluation suites and inspect the results). Install the extra first:
 
 ```bash
-uv run python examples/webui/presets_webui.py
+uv sync --extra webui            # add --extra sqlite for Evaluations
 ```
 
-Step 3. Open the browser:
-
-```text
-http://127.0.0.1:8080
-```
+### Option 1: Sample apps
 
-If auto-open is enabled in your environment, the browser may open automatically.
-
-### Option 2: Run the WebUI against your current project directory
-
-Use this when you want Agentbyte to scan a directory for exported `agent`, `workflow`, or `orchestrator` objects.
-
-Important: discovery is convention-based. The scanned directory must contain Python modules that expose top-level variables literally named `agent`, `workflow`, or `orchestrator`. If you point `--dir` at a folder that does not export those names, the UI will load but show `No entities found`.
-
-Step 1. Install the WebUI extra:
+No API key needed: three demo agents, a team and a workflow on a fake model client.
 
 ```bash
-uv sync --extra webui
+uv run python examples/webui/in_memory.py                          # http://127.0.0.1:8070
+uv run python examples/webui/in_memory.py --eval-history eval.sqlite   # plus Evaluations
 ```
 
-Step 2. Launch the WebUI and scan the current directory:
+With your OpenAI or Azure OpenAI credentials, the preset agents, teams and workflow:
 
 ```bash
-uv run agentbyte webui --dir .
+uv sync --extra webui --extra openai        # or --extra azureopenai
+uv run python examples/webui/presets_webui.py                      # http://127.0.0.1:8080
+uv run python examples/webui/presets_webui.py --eval-history eval.sqlite
 ```
 
-Step 3. Open the browser:
+Both launchers accept `--port`, `--no-open`, `--eval-history PATH` and `--eval-catalog CATALOG`
+(default: the example suites in `examples/eval/workbench_catalog.py`).
 
-```text
-http://127.0.0.1:8080
-```
+### Option 2: Evaluations for your suites
 
-Useful variants:
+`agentbyte eval ui` opens the Evaluations pages on their own (no Playground). Point it at the
+Python file that defines your `EvalSuiteRegistry`:
 
 ```bash
-uv run agentbyte webui --dir . --port 8080 --host 127.0.0.1 --no-open
-uv run agentbyte webui --dir examples --port 8090
-# evaluation API under /api/eval (needs the sqlite extra; default suite: built-in smoke)
-uv run agentbyte webui --eval-history ./eval.sqlite --eval-catalog myapp.evaluations:registry
+uv sync --extra webui --extra sqlite
+uv run agentbyte eval ui --catalog examples/eval/workbench_catalog.py      # http://127.0.0.1:8090
 ```
 
-For this repository, the most reliable first-run path is the preset-backed launcher:
+- `--catalog` takes a `.py` file or a module path (`myapp.evaluations`); add `:name` only when the
+  file defines more than one registry. Without `--catalog` it serves the built-in smoke suite.
+- Runs are stored in `./eval.sqlite` unless you pass `--history PATH`. Only one process can write
+  a history file: stop the UI before `agentbyte eval run` on the same file, or use another file.
+- The same catalog works in the terminal: `uv run agentbyte eval run capitals --catalog examples/eval/workbench_catalog.py`.
+- To serve agents and Evaluations together: `agentbyte webui --dir my_entities --eval-catalog myapp/evaluations.py`.
 
-```bash
-uv run python examples/webui/presets_webui.py
-```
+Start with the end-to-end guide, `docs/guides/evaluation-end-to-end.md` (dataset → suite → history → UI), and the
+notebook `notebooks/usecases/eval/06-eval-end-to-end.ipynb` (real OpenAI/Azure model). Details: `docs/study/05-eval_topics/`.
 
-Use `agentbyte webui --dir ...` when you have a directory of exportable demo modules, for example:
+### Option 3: Scan a project directory
+
+`agentbyte webui --dir PATH` loads entities from Python files. Discovery is convention-based: each
+module (or package) must define a top-level variable named `agent`, `orchestrator` or `workflow`
+holding an Agentbyte object. Folders without those names show "No agents, orchestrators, or
+workflows found"; this is the case for `examples/`, which uses the launchers above instead.
 
 ```python
-# my_entities.py
-agent = ...
-workflow = ...
-orchestrator = ...
+# my_entities/support.py
+agent = Agent(name="support", ...)
 ```
 
-### Option 3: Run it programmatically
+```bash
+uv run agentbyte webui --dir my_entities
+uv run agentbyte webui --dir my_entities --eval-catalog myapp/evaluations.py   # both sections
+```
 
-Use this when you want to serve in-memory entities directly from Python.
+### Option 4: From Python
 
 ```python
 from agentbyte.webui import serve
 
-serve(entities=[agent], port=8080, auto_open=True)
+serve(entities=[agent], port=8080)
+serve(entities=[agent], eval_registry=registry, eval_history="eval.sqlite")   # with Evaluations
 ```
 
 ### Quick Troubleshooting
 
-If the app does not start:
-
-```bash
-uv sync --extra webui
-```
-
-If port `8080` is already in use:
-
-```bash
-uv run agentbyte webui --dir . --port 8090
-```
-
-If you do not want the browser to open automatically:
-
-```bash
-uv run agentbyte webui --dir . --no-open
-```
+- Port in use: add `--port 8090`. The UI works on any port.
+- No browser wanted: add `--no-open`.
+- "No agents, orchestrators, or workflows found": the `--dir` folder has no module with a top-level
+  `agent`, `orchestrator` or `workflow`; use a launcher from Option 1 to see the samples.
+- No Evaluations switch in the header: start `agentbyte webui` with `--eval-catalog`, or use `agentbyte eval ui`.
+- After upgrading, hard-reload the page (Cmd/Ctrl-Shift-R) to load the new UI.
 
 ## Development
 

{agentbyte-0.33.0 → agentbyte-0.34.0}/README.md

@@ -2,7 +2,7 @@
 
 Agentbyte is an observability-first agentic AI framework for building and studying multiagent systems with a learning-first, implementation-oriented workflow.
 
-Current release: **0.33.0**
+Current release: **0.34.0**
 
 ## Building an Agent
 
@@ -253,110 +253,87 @@ uv sync --all-extras --all-groups
 
 ## Run The WebUI
 
-### Option 1: Run the preset-backed app
-
-This is the easiest way to see the WebUI working end to end with real preset entities:
-- preset agents
-- preset orchestrators
-- preset workflow
-
-Step 1. Install the WebUI extra:
-
-```bash
-uv sync --extra webui
-```
-
-Step 2. Start the preset-backed app:
+The WebUI has two sections: the **Playground** (chat with agents, orchestrators and workflows) and
+**Evaluations** (run registered evaluation suites and inspect the results). Install the extra first:
 
 ```bash
-uv run python examples/webui/presets_webui.py
+uv sync --extra webui            # add --extra sqlite for Evaluations
 ```
 
-Step 3. Open the browser:
-
-```text
-http://127.0.0.1:8080
-```
+### Option 1: Sample apps
 
-If auto-open is enabled in your environment, the browser may open automatically.
-
-### Option 2: Run the WebUI against your current project directory
-
-Use this when you want Agentbyte to scan a directory for exported `agent`, `workflow`, or `orchestrator` objects.
-
-Important: discovery is convention-based. The scanned directory must contain Python modules that expose top-level variables literally named `agent`, `workflow`, or `orchestrator`. If you point `--dir` at a folder that does not export those names, the UI will load but show `No entities found`.
-
-Step 1. Install the WebUI extra:
+No API key needed: three demo agents, a team and a workflow on a fake model client.
 
 ```bash
-uv sync --extra webui
+uv run python examples/webui/in_memory.py                          # http://127.0.0.1:8070
+uv run python examples/webui/in_memory.py --eval-history eval.sqlite   # plus Evaluations
 ```
 
-Step 2. Launch the WebUI and scan the current directory:
+With your OpenAI or Azure OpenAI credentials, the preset agents, teams and workflow:
 
 ```bash
-uv run agentbyte webui --dir .
+uv sync --extra webui --extra openai        # or --extra azureopenai
+uv run python examples/webui/presets_webui.py                      # http://127.0.0.1:8080
+uv run python examples/webui/presets_webui.py --eval-history eval.sqlite
 ```
 
-Step 3. Open the browser:
+Both launchers accept `--port`, `--no-open`, `--eval-history PATH` and `--eval-catalog CATALOG`
+(default: the example suites in `examples/eval/workbench_catalog.py`).
 
-```text
-http://127.0.0.1:8080
-```
+### Option 2: Evaluations for your suites
 
-Useful variants:
+`agentbyte eval ui` opens the Evaluations pages on their own (no Playground). Point it at the
+Python file that defines your `EvalSuiteRegistry`:
 
 ```bash
-uv run agentbyte webui --dir . --port 8080 --host 127.0.0.1 --no-open
-uv run agentbyte webui --dir examples --port 8090
-# evaluation API under /api/eval (needs the sqlite extra; default suite: built-in smoke)
-uv run agentbyte webui --eval-history ./eval.sqlite --eval-catalog myapp.evaluations:registry
+uv sync --extra webui --extra sqlite
+uv run agentbyte eval ui --catalog examples/eval/workbench_catalog.py      # http://127.0.0.1:8090
 ```
 
-For this repository, the most reliable first-run path is the preset-backed launcher:
+- `--catalog` takes a `.py` file or a module path (`myapp.evaluations`); add `:name` only when the
+  file defines more than one registry. Without `--catalog` it serves the built-in smoke suite.
+- Runs are stored in `./eval.sqlite` unless you pass `--history PATH`. Only one process can write
+  a history file: stop the UI before `agentbyte eval run` on the same file, or use another file.
+- The same catalog works in the terminal: `uv run agentbyte eval run capitals --catalog examples/eval/workbench_catalog.py`.
+- To serve agents and Evaluations together: `agentbyte webui --dir my_entities --eval-catalog myapp/evaluations.py`.
 
-```bash
-uv run python examples/webui/presets_webui.py
-```
+Start with the end-to-end guide, `docs/guides/evaluation-end-to-end.md` (dataset → suite → history → UI), and the
+notebook `notebooks/usecases/eval/06-eval-end-to-end.ipynb` (real OpenAI/Azure model). Details: `docs/study/05-eval_topics/`.
 
-Use `agentbyte webui --dir ...` when you have a directory of exportable demo modules, for example:
+### Option 3: Scan a project directory
+
+`agentbyte webui --dir PATH` loads entities from Python files. Discovery is convention-based: each
+module (or package) must define a top-level variable named `agent`, `orchestrator` or `workflow`
+holding an Agentbyte object. Folders without those names show "No agents, orchestrators, or
+workflows found"; this is the case for `examples/`, which uses the launchers above instead.
 
 ```python
-# my_entities.py
-agent = ...
-workflow = ...
-orchestrator = ...
+# my_entities/support.py
+agent = Agent(name="support", ...)
 ```
 
-### Option 3: Run it programmatically
+```bash
+uv run agentbyte webui --dir my_entities
+uv run agentbyte webui --dir my_entities --eval-catalog myapp/evaluations.py   # both sections
+```
 
-Use this when you want to serve in-memory entities directly from Python.
+### Option 4: From Python
 
 ```python
 from agentbyte.webui import serve
 
-serve(entities=[agent], port=8080, auto_open=True)
+serve(entities=[agent], port=8080)
+serve(entities=[agent], eval_registry=registry, eval_history="eval.sqlite")   # with Evaluations
 ```
 
 ### Quick Troubleshooting
 
-If the app does not start:
-
-```bash
-uv sync --extra webui
-```
-
-If port `8080` is already in use:
-
-```bash
-uv run agentbyte webui --dir . --port 8090
-```
-
-If you do not want the browser to open automatically:
-
-```bash
-uv run agentbyte webui --dir . --no-open
-```
+- Port in use: add `--port 8090`. The UI works on any port.
+- No browser wanted: add `--no-open`.
+- "No agents, orchestrators, or workflows found": the `--dir` folder has no module with a top-level
+  `agent`, `orchestrator` or `workflow`; use a launcher from Option 1 to see the samples.
+- No Evaluations switch in the header: start `agentbyte webui` with `--eval-catalog`, or use `agentbyte eval ui`.
+- After upgrading, hard-reload the page (Cmd/Ctrl-Shift-R) to load the new UI.
 
 ## Development
 

agentbyte-0.34.0/src/agentbyte/__about__.py

@@ -0,0 +1,2 @@
+__version__ = "0.34.0"
+VERSION = __version__

{agentbyte-0.33.0 → agentbyte-0.34.0}/src/agentbyte/cli/eval_commands.py

@@ -19,7 +19,9 @@ EXIT_FAILED = 1
 EXIT_ERROR = 2
 EXIT_INTERRUPTED = 130
 
-DEFAULT_CATALOG = "agentbyte.eval.workbench.smoke:registry"
+DEFAULT_CATALOG = "agentbyte.eval.workbench.smoke"
+DEFAULT_HISTORY = "eval.sqlite"
+DEFAULT_UI_PORT = 8090
 
 
 class _CliError(Exception):
@@ -32,9 +34,11 @@ def add_eval_parser(subparsers: Any) -> None:
     eval_sub = eval_parser.add_subparsers(dest="eval_command", metavar="<subcommand>")
 
     catalog_help = (
-        "Import path to an EvalSuiteRegistry in 'module.path:object' format "
+        "Where the EvalSuiteRegistry is: a .py file (myapp/evaluations.py) or a module "
+        "(myapp.evaluations); add ':name' if it defines more than one "
         f"(default: the built-in smoke suite, {DEFAULT_CATALOG})"
     )
+    history_help = f"SQLite evaluation history file (default: ./{DEFAULT_HISTORY})"
 
     list_parser = eval_sub.add_parser("list", help="List the suites in a catalog")
     list_parser.add_argument("--catalog", default=DEFAULT_CATALOG, help=catalog_help)
@@ -43,7 +47,7 @@ def add_eval_parser(subparsers: Any) -> None:
     run_parser = eval_sub.add_parser("run", help="Run one suite and record it")
     run_parser.add_argument("suite", help="Registered suite name")
     run_parser.add_argument("--catalog", default=DEFAULT_CATALOG, help=catalog_help)
-    run_parser.add_argument("--history", required=True, help="SQLite evaluation history file")
+    run_parser.add_argument("--history", default=DEFAULT_HISTORY, help=history_help)
     run_parser.add_argument(
         "--capture",
         choices=["summary", "full"],
@@ -55,16 +59,27 @@ def add_eval_parser(subparsers: Any) -> None:
 
     results_parser = eval_sub.add_parser("results", help="Show the verdict of a recorded run")
     results_parser.add_argument("run_id", help="Run ID printed by 'agentbyte eval run'")
-    results_parser.add_argument("--history", required=True, help="SQLite evaluation history file")
+    results_parser.add_argument("--history", default=DEFAULT_HISTORY, help=history_help)
     results_parser.add_argument("--format", choices=["text", "json"], default="text")
 
+    ui_parser = eval_sub.add_parser(
+        "ui", help="Open the evaluation pages in the browser (Evaluations only, no Playground)"
+    )
+    ui_parser.add_argument("--catalog", default=DEFAULT_CATALOG, help=catalog_help)
+    ui_parser.add_argument("--history", default=DEFAULT_HISTORY, help=history_help)
+    ui_parser.add_argument("--port", "-p", type=int, default=DEFAULT_UI_PORT,
+                           help=f"Port to serve on (default: {DEFAULT_UI_PORT})")
+    ui_parser.add_argument("--host", default="127.0.0.1", help="Host to bind (default: 127.0.0.1)")
+    ui_parser.add_argument("--no-open", action="store_true", help="Do not open the browser")
+    ui_parser.add_argument("--log-level", choices=["debug", "info", "warning", "error"], default="warning")
+
 
 def cmd_eval(args: argparse.Namespace) -> int:
     """Dispatch ``agentbyte eval <subcommand>``."""
-    commands = {"list": _cmd_list, "run": _cmd_run, "results": _cmd_results}
+    commands = {"list": _cmd_list, "run": _cmd_run, "results": _cmd_results, "ui": _cmd_ui}
     command = commands.get(getattr(args, "eval_command", None) or "")
     if command is None:
-        print("usage: agentbyte eval {list,run,results} ...", file=sys.stderr)
+        print("usage: agentbyte eval {list,run,results,ui} ...", file=sys.stderr)
         return EXIT_ERROR
     try:
         return command(args)
@@ -73,6 +88,49 @@ def cmd_eval(args: argparse.Namespace) -> int:
         return EXIT_ERROR
 
 
+# -- ui ---------------------------------------------------------------------------
+
+
+def _cmd_ui(args: argparse.Namespace) -> int:
+    """Serve the Evaluations pages only; returns when the server stops."""
+    from agentbyte._optional import MissingExtraError, require
+
+    registry = load_registry(args.catalog)
+    try:
+        require("aiosqlite", extra="sqlite", feature="Evaluation history")
+        from agentbyte.webui import launch
+    except (MissingExtraError, ImportError) as exc:
+        raise _CliError(str(exc)) from exc
+    asyncio.run(_check_history_is_free(args.history))
+    print(f"Evaluations: http://{args.host}:{args.port}  (history {args.history}; Ctrl-C to stop)", file=sys.stderr)
+    try:
+        launch(
+            port=args.port,
+            host=args.host,
+            auto_open=not args.no_open,
+            log_level=args.log_level,
+            eval_registry=registry,
+            eval_history=args.history,
+            playground=False,
+        )
+    except KeyboardInterrupt:
+        return EXIT_PASSED
+    except Exception as exc:  # boundary: report server start-up failures (port in use, locked history)
+        raise _CliError(f"Could not start the evaluation UI: {exc}") from exc
+    return EXIT_PASSED
+
+
+async def _check_history_is_free(path: str) -> None:
+    """Fail fast when another process (a CLI run, another server) holds the history file."""
+    from agentbyte.eval.history import EvalHistoryError
+
+    try:
+        async with _open_store(path, writer=True):
+            pass
+    except EvalHistoryError as exc:
+        raise _CliError(f"{exc} (only one process can write {path!r} at a time)") from exc
+
+
 # -- list -------------------------------------------------------------------------
 
 
@@ -188,7 +246,7 @@ def load_registry(import_path: str) -> Any:
     from agentbyte.eval.workbench import EvalSuiteRegistry
 
     try:
-        return import_object(import_path, EvalSuiteRegistry, example=DEFAULT_CATALOG)
+        return import_object(import_path, EvalSuiteRegistry, example="myapp/evaluations.py")
     except ObjectImportError as exc:
         raise _CliError(str(exc)) from exc
 

agentbyte-0.34.0/src/agentbyte/cli/loading.py

@@ -0,0 +1,115 @@
+"""Load trusted objects named on the command line (catalogs, registries)."""
+
+from __future__ import annotations
+
+import hashlib
+import importlib
+import importlib.util
+import os
+import sys
+from pathlib import Path
+from types import ModuleType
+from typing import TypeVar
+
+T = TypeVar("T")
+
+
+class ObjectImportError(ValueError):
+    """A command-line reference could not be loaded as the expected type."""
+
+
+def import_object(reference: str, expected_type: type[T], *, example: str | None = None) -> T:
+    """Load an ``expected_type`` instance from a module or a Python file.
+
+    ``reference`` is ``SOURCE`` or ``SOURCE:NAME``, where ``SOURCE`` is a dotted module
+    path (``myapp.evaluations``) or a ``.py`` file path (``myapp/evaluations.py``).
+    Without ``:NAME`` the module must define exactly one ``expected_type`` instance.
+    Like ``python -m``, modules in the current working directory are importable.
+
+    Only for trusted configuration such as CLI arguments: importing runs module code.
+    Every failure raises ``ObjectImportError`` with a message suitable for the user.
+    """
+    kind = expected_type.__name__
+    hint = f" Example: {example}" if example else ""
+    source, name = _split(reference)
+    if not source or name == "":
+        raise ObjectImportError(
+            f"Expected 'module.path', 'path/to/file.py' or either with ':object_name' for a {kind}, "
+            f"got {reference!r}.{hint}"
+        )
+    module = _load_file(source) if _is_file_reference(source) else _load_module(source)
+    if name is None:
+        return _only_instance(module, source, expected_type)
+    if not hasattr(module, name):
+        raise ObjectImportError(f"{name!r} not found in {source!r}.")
+    value = getattr(module, name)
+    if not isinstance(value, expected_type):
+        raise ObjectImportError(f"{reference!r} is a {type(value).__name__!r}, not a {kind} instance.")
+    return value
+
+
+def _split(reference: str) -> tuple[str, str | None]:
+    """``SOURCE`` and the optional ``NAME``; a Windows drive colon is not a separator."""
+    if ":" not in reference:
+        return reference, None
+    source, name = reference.rsplit(":", 1)
+    if any(sep in name for sep in ("/", "\\")) or name.endswith(".py"):
+        return reference, None
+    return source, name
+
+
+def _is_file_reference(source: str) -> bool:
+    return source.endswith(".py") or "/" in source or "\\" in source
+
+
+def _load_module(module_path: str) -> ModuleType:
+    _import_from_working_directory()
+    try:
+        return importlib.import_module(module_path)
+    except Exception as exc:  # boundary: any import-time failure is a configuration error
+        raise ObjectImportError(f"Could not import module {module_path!r}: {exc}") from exc
+
+
+def _load_file(file_path: str) -> ModuleType:
+    path = Path(file_path).expanduser().resolve()
+    if path.suffix != ".py" or not path.is_file():
+        raise ObjectImportError(f"No Python file at {file_path!r}.")
+    # One stable module name per file, so loading the same file twice reuses it.
+    digest = hashlib.sha1(str(path).encode()).hexdigest()[:12]
+    module_name = f"_agentbyte_cli_{path.stem}_{digest}"
+    if module_name in sys.modules:
+        return sys.modules[module_name]
+    _import_from_working_directory()
+    spec = importlib.util.spec_from_file_location(module_name, path)
+    if spec is None or spec.loader is None:
+        raise ObjectImportError(f"Could not load {file_path!r} as a Python module.")
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[module_name] = module
+    try:
+        spec.loader.exec_module(module)
+    except Exception as exc:  # boundary: any import-time failure is a configuration error
+        del sys.modules[module_name]
+        raise ObjectImportError(f"Could not load {file_path!r}: {exc}") from exc
+    return module
+
+
+def _only_instance(module: ModuleType, source: str, expected_type: type[T]) -> T:
+    found = {
+        name: value
+        for name, value in vars(module).items()
+        if not name.startswith("__") and isinstance(value, expected_type)
+    }
+    unique = {id(value): name for name, value in found.items()}
+    if len(unique) == 1:
+        return next(iter(found.values()))
+    kind = expected_type.__name__
+    if not found:
+        raise ObjectImportError(f"No {kind} found in {source!r}.")
+    names = ", ".join(sorted(found))
+    raise ObjectImportError(f"{source!r} defines several {kind} objects ({names}); add ':name' to pick one.")
+
+
+def _import_from_working_directory() -> None:
+    cwd = os.getcwd()
+    if cwd not in sys.path and "" not in sys.path:
+        sys.path.insert(0, cwd)