agentbyte 0.32.2__tar.gz → 0.33.0__tar.gz

{agentbyte-0.32.2 → agentbyte-0.33.0}/CHANGELOG.md

@@ -6,6 +6,36 @@ The format follows Keep a Changelog principles and semantic versioning.
 
 ## [Unreleased]
 
+## [0.33.0] - 2026-10-01
+
+Upgrade notes (breaking):
+
+- Removed the `test`, `dev` and `coding` extras; they only held contributor tooling. Contributors use `uv sync --all-extras --all-groups`.
+- S3 datasets use native async `aioboto3` (`aws` extra) instead of synchronous `boto3`. `DatasetWriter.write_files`/`dataset_exists`, `DatasetPublisher.publish_dataset` and `agentbyte.dataset.publish_dataset` are now coroutines: `await` them, and make custom writers/publishers `async def`. A `client=` passed to `S3DatasetSource`, `S3DatasetWriter` or `S3DatasetPublisher` must be an async S3 client; it is used as-is and never closed.
+- A missing optional dependency now raises `agentbyte.MissingExtraError` (a `ModuleNotFoundError`) everywhere. The WebUI and `WorkflowVisualizer.to_dot()` previously raised `RuntimeError`.
+
+### Added
+
+- Evaluation workbench, step 1 (spec 0052): `agentbyte.eval.workbench` with `EvalSuite`, `EvalSuiteRegistry`, `EvalSuiteCoordinator` and pass/fail verdicts (`SuiteRunSummary`); an item passes only when execution completed, the judge scored it and `overall >= min_score`. New CLI `agentbyte eval list|run|results` (exit codes 0 passed, 1 failed/incomplete, 2 configuration/storage error, 130 interrupted) and a packaged, provider-free `smoke` suite. `EvalRunRecorder.evaluate`/`evaluate_stream` accept `on_start`. `python -m agentbyte.cli` works.
+- Evaluation workbench, step 2 (spec 0052): HTTP job lifecycle. `EvalJobManager` runs suites in the background (one active, ten queued); admission is idempotent per owner and key and atomic in the store, a full queue is rejected without a record, cancel is idempotent, shutdown commits owned jobs as `cancelled` and startup marks leftovers `interrupted`. WebUI routes under `/api/eval` (suites, runs, items, SSE `snapshot` events keyed by run revision, cancel), enabled by `AgentbyteWebUIServer(eval_store=... | eval_history=..., eval_registry=...)` or `agentbyte webui --eval-history PATH [--eval-catalog module:registry]`. `EvalRunRecorder.admit()`/`execute_stream()` split admission from execution; `SuiteRunSummary.revision`.
+- Evaluation workbench docs: study guide `docs/study/05-eval_topics/05_topic_eval_workbench.md`, offline example catalog `examples/eval/workbench_catalog.py` and notebook `notebooks/usecases/eval/05-eval-workbench.ipynb`.
+- `tiktoken` extra for `TiktokenCounter`.
+- `agentbyte.MissingExtraError`: one error type and message format ("<feature> needs the '<extra>' extra: pip install 'agentbyte[<extra>]'") for every optional dependency, via `agentbyte._optional.require`.
+- `agentbyte.cli.loading.import_object(reference, expected_type)`: one loader for `module.path:object` CLI arguments.
+
+### Changed
+
+- CLI `--catalog` / `registry check` references are imported like `python -m`: modules in the current working directory are found without installing them.
+- Evaluation history idempotency: reusing a key now also requires the same capture, criteria, labels, judge and target fingerprint; otherwise `EvalHistoryConflictError`.
+- README "Installation" is now one table of extras, what they install and which features need them.
+- The S3 dataset importer no longer wraps writer calls in `asyncio.to_thread`.
+- `all` (and `sql`) are composed from the other extras by reference, so they can no longer drift. Dependency groups now hold only local tooling (`test`, `dev`, `experiment`); groups that mirrored extras and the unused `extlib` group (Microsoft Agent Framework packages) were removed.
+
+### Fixed
+
+- SQLite datasets (`sqlite`, `sqlite_db` engines) no longer keep a script from exiting when they are not closed: their cached aiosqlite connection runs on a daemon thread. `aclose()` / `async with` remain the way to release them promptly.
+- Use-case notebooks (eval 01–04, workflow 01–02) updated for the current async dataset, eval-dataset and judge-integrity APIs; the repository's `contracting/metadata26` eval dataset was re-saved with the current schema (`metadata` column). OTel install hints now say `uv sync --extra otel`.
+
 ## [0.32.2] - 2026-10-01
 
 ### Fixed

{agentbyte-0.32.2 → agentbyte-0.33.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: agentbyte
-Version: 0.32.2
+Version: 0.33.0
 Summary: A toolkit for designing multiagent systems
 Author-email: MrDataPsycho <mr.data.psycho@gmail.com>
 License-Expression: LicenseRef-Proprietary
@@ -27,22 +27,14 @@ Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.39.1; extra == 'all'
 Requires-Dist: opentelemetry-sdk>=1.39.1; extra == 'all'
 Requires-Dist: sqlalchemy[asyncio]>=2.0.43; extra == 'all'
 Requires-Dist: sqlmodel>=0.0.38; extra == 'all'
-Requires-Dist: uvicorn>=0.42.0; extra == 'all'
+Requires-Dist: tiktoken>=0.9.0; extra == 'all'
+Requires-Dist: uvicorn[standard]>=0.44.0; extra == 'all'
 Requires-Dist: youtube-transcript-api>=0.6; extra == 'all'
 Provides-Extra: aws
 Requires-Dist: aioboto3>=15.5.0; extra == 'aws'
 Provides-Extra: azureopenai
 Requires-Dist: azure-identity>=1.25.1; extra == 'azureopenai'
 Requires-Dist: openai>=1.107.1; extra == 'azureopenai'
-Provides-Extra: coding
-Provides-Extra: dev
-Requires-Dist: ipykernel>=7.1.0; extra == 'dev'
-Requires-Dist: nbclient>=0.10.4; extra == 'dev'
-Requires-Dist: nbformat>=5.10.4; extra == 'dev'
-Requires-Dist: picoagents>=0.4.0; extra == 'dev'
-Requires-Dist: pydantic-settings>=2.13.0; extra == 'dev'
-Requires-Dist: pymupdf-layout>=1.26.6; extra == 'dev'
-Requires-Dist: pymupdf4llm>=0.2.9; extra == 'dev'
 Provides-Extra: openai
 Requires-Dist: openai>=1.107.1; extra == 'openai'
 Provides-Extra: optim
@@ -67,18 +59,8 @@ Requires-Dist: sqlalchemy[asyncio]>=2.0.43; extra == 'sql'
 Requires-Dist: sqlmodel>=0.0.38; extra == 'sql'
 Provides-Extra: sqlite
 Requires-Dist: aiosqlite>=0.22.1; extra == 'sqlite'
-Provides-Extra: test
-Requires-Dist: aiosqlite>=0.22.1; extra == 'test'
-Requires-Dist: arxiv>=2.1; extra == 'test'
-Requires-Dist: beautifulsoup4>=4.12; extra == 'test'
-Requires-Dist: greenlet>=3.0.0; extra == 'test'
-Requires-Dist: html2text>=2024.2; extra == 'test'
-Requires-Dist: pytest-asyncio>=1.3.0; extra == 'test'
-Requires-Dist: pytest-cov>=7.0.0; extra == 'test'
-Requires-Dist: pytest>=9.0.1; extra == 'test'
-Requires-Dist: ruff>=0.15.0; extra == 'test'
-Requires-Dist: sqlmodel>=0.0.38; extra == 'test'
-Requires-Dist: youtube-transcript-api>=0.6; extra == 'test'
+Provides-Extra: tiktoken
+Requires-Dist: tiktoken>=0.9.0; extra == 'tiktoken'
 Provides-Extra: viz
 Requires-Dist: graphviz>=0.21; extra == 'viz'
 Provides-Extra: webui
@@ -90,12 +72,16 @@ Description-Content-Type: text/markdown
 
 Agentbyte is an observability-first agentic AI framework for building and studying multiagent systems with a learning-first, implementation-oriented workflow.
 
-Current release: **0.32.2**
+Current release: **0.33.0**
 
 ## Building an Agent
 
 Every example below builds on the same domain: a customer **support agent**. Start with a plain agent and a model client — no tools, no middleware:
 
+```bash
+pip install "agentbyte[openai]"   # the model client lives in an extra; see Install below
+```
+
 ```python
 from agentbyte import Agent
 from agentbyte.llm import OpenAIChatCompletionClient
@@ -302,56 +288,39 @@ Reading a trace: `chat gpt-4.1-mini` spans show **per-call** usage/cost/finish r
 
 Python requirement: **3.11+**
 
-```bash
-uv sync --all-groups
-```
-
-Optional extras:
-
-```bash
-uv sync --extra openai        # OpenAI client
-uv sync --extra azureopenai   # Azure OpenAI client + azure-identity
-uv sync --extra aws           # S3 datasets (aioboto3)
-uv sync --extra otel          # OpenTelemetry export
-uv sync --extra webui         # WebUI playground (FastAPI)
-uv sync --extra sqlite        # SQLite stores: datasets, workflow checkpoints, eval history
-uv sync --extra postgres      # PostgreSQL stores: usage logging, eval history
-uv sync --extra sql           # sqlite + postgres
-uv sync --extra optim         # prompt optimization (GEPA)
-uv sync --extra viz           # workflow diagrams (graphviz)
-uv sync --extra research      # research tools
-```
-
-### Install in another project (pip / uv add)
-
-Use extras to enable provider + telemetry support:
-
-```bash
-pip install "agentbyte[azureopenai,otel]"
-```
-
-```bash
-uv add "agentbyte[azureopenai,otel]"
-```
-
-For the browser WebUI:
+The base package (`pip install agentbyte`) has no model provider or storage driver. Add the
+extras for the features you use. Using a feature without its extra raises
+`MissingExtraError` (an `ImportError`) that names the extra to install.
 
 ```bash
-pip install "agentbyte[webui]"
+pip install "agentbyte[openai,sqlite]"
 # or
-uv add "agentbyte[webui]"
+uv add "agentbyte[openai,sqlite]"
 ```
 
-Install all optional features:
+| Extra | Installs | Needed for |
+| --- | --- | --- |
+| `openai` | `openai` | `OpenAIChatCompletionClient`, `OpenAIEmbeddingClient` |
+| `azureopenai` | `openai`, `azure-identity` | Azure OpenAI clients and Entra ID credentials |
+| `aws` | `aioboto3` | S3 dataset sources, writers and publishers |
+| `otel` | OpenTelemetry API, SDK, OTLP exporter | OpenTelemetry tracing and metrics |
+| `webui` | `fastapi`, `uvicorn` | `agentbyte webui` |
+| `sqlite` | `aiosqlite` | SQLite datasets, workflow checkpoints, evaluation history |
+| `postgres` | `sqlalchemy[asyncio]`, `sqlmodel`, `asyncpg` | `SqlUsageBackend`, `SqlEvalRunStore` |
+| `sql` | `sqlite` + `postgres` | Both SQL backends |
+| `tiktoken` | `tiktoken` | `TiktokenCounter` for exact compaction token counts |
+| `optim` | `gepa` | `optimize_with_gepa()` |
+| `viz` | `graphviz` | `WorkflowVisualizer.to_dot()` |
+| `research` | `beautifulsoup4`, `html2text`, `arxiv`, `youtube-transcript-api` | `agentbyte.tools.research_tools` |
+| `all` | every extra above | Everything |
+
+Contributors working in this repository install every extra plus the local tooling groups
+(`test`, `dev`, `experiment`):
 
 ```bash
-pip install "agentbyte[all]"
-# or
-uv add "agentbyte[all]"
+uv sync --all-extras --all-groups
 ```
 
-Note: the Azure extra is `azureopenai`.
-
 ## Run The WebUI
 
 ### Option 1: Run the preset-backed app
@@ -410,6 +379,8 @@ Useful variants:
 ```bash
 uv run agentbyte webui --dir . --port 8080 --host 127.0.0.1 --no-open
 uv run agentbyte webui --dir examples --port 8090
+# evaluation API under /api/eval (needs the sqlite extra; default suite: built-in smoke)
+uv run agentbyte webui --eval-history ./eval.sqlite --eval-catalog myapp.evaluations:registry
 ```
 
 For this repository, the most reliable first-run path is the preset-backed launcher:

{agentbyte-0.32.2 → agentbyte-0.33.0}/README.md

@@ -2,12 +2,16 @@
 
 Agentbyte is an observability-first agentic AI framework for building and studying multiagent systems with a learning-first, implementation-oriented workflow.
 
-Current release: **0.32.2**
+Current release: **0.33.0**
 
 ## Building an Agent
 
 Every example below builds on the same domain: a customer **support agent**. Start with a plain agent and a model client — no tools, no middleware:
 
+```bash
+pip install "agentbyte[openai]"   # the model client lives in an extra; see Install below
+```
+
 ```python
 from agentbyte import Agent
 from agentbyte.llm import OpenAIChatCompletionClient
@@ -214,56 +218,39 @@ Reading a trace: `chat gpt-4.1-mini` spans show **per-call** usage/cost/finish r
 
 Python requirement: **3.11+**
 
-```bash
-uv sync --all-groups
-```
-
-Optional extras:
-
-```bash
-uv sync --extra openai        # OpenAI client
-uv sync --extra azureopenai   # Azure OpenAI client + azure-identity
-uv sync --extra aws           # S3 datasets (aioboto3)
-uv sync --extra otel          # OpenTelemetry export
-uv sync --extra webui         # WebUI playground (FastAPI)
-uv sync --extra sqlite        # SQLite stores: datasets, workflow checkpoints, eval history
-uv sync --extra postgres      # PostgreSQL stores: usage logging, eval history
-uv sync --extra sql           # sqlite + postgres
-uv sync --extra optim         # prompt optimization (GEPA)
-uv sync --extra viz           # workflow diagrams (graphviz)
-uv sync --extra research      # research tools
-```
-
-### Install in another project (pip / uv add)
-
-Use extras to enable provider + telemetry support:
-
-```bash
-pip install "agentbyte[azureopenai,otel]"
-```
-
-```bash
-uv add "agentbyte[azureopenai,otel]"
-```
-
-For the browser WebUI:
+The base package (`pip install agentbyte`) has no model provider or storage driver. Add the
+extras for the features you use. Using a feature without its extra raises
+`MissingExtraError` (an `ImportError`) that names the extra to install.
 
 ```bash
-pip install "agentbyte[webui]"
+pip install "agentbyte[openai,sqlite]"
 # or
-uv add "agentbyte[webui]"
+uv add "agentbyte[openai,sqlite]"
 ```
 
-Install all optional features:
+| Extra | Installs | Needed for |
+| --- | --- | --- |
+| `openai` | `openai` | `OpenAIChatCompletionClient`, `OpenAIEmbeddingClient` |
+| `azureopenai` | `openai`, `azure-identity` | Azure OpenAI clients and Entra ID credentials |
+| `aws` | `aioboto3` | S3 dataset sources, writers and publishers |
+| `otel` | OpenTelemetry API, SDK, OTLP exporter | OpenTelemetry tracing and metrics |
+| `webui` | `fastapi`, `uvicorn` | `agentbyte webui` |
+| `sqlite` | `aiosqlite` | SQLite datasets, workflow checkpoints, evaluation history |
+| `postgres` | `sqlalchemy[asyncio]`, `sqlmodel`, `asyncpg` | `SqlUsageBackend`, `SqlEvalRunStore` |
+| `sql` | `sqlite` + `postgres` | Both SQL backends |
+| `tiktoken` | `tiktoken` | `TiktokenCounter` for exact compaction token counts |
+| `optim` | `gepa` | `optimize_with_gepa()` |
+| `viz` | `graphviz` | `WorkflowVisualizer.to_dot()` |
+| `research` | `beautifulsoup4`, `html2text`, `arxiv`, `youtube-transcript-api` | `agentbyte.tools.research_tools` |
+| `all` | every extra above | Everything |
+
+Contributors working in this repository install every extra plus the local tooling groups
+(`test`, `dev`, `experiment`):
 
 ```bash
-pip install "agentbyte[all]"
-# or
-uv add "agentbyte[all]"
+uv sync --all-extras --all-groups
 ```
 
-Note: the Azure extra is `azureopenai`.
-
 ## Run The WebUI
 
 ### Option 1: Run the preset-backed app
@@ -322,6 +309,8 @@ Useful variants:
 ```bash
 uv run agentbyte webui --dir . --port 8080 --host 127.0.0.1 --no-open
 uv run agentbyte webui --dir examples --port 8090
+# evaluation API under /api/eval (needs the sqlite extra; default suite: built-in smoke)
+uv run agentbyte webui --eval-history ./eval.sqlite --eval-catalog myapp.evaluations:registry
 ```
 
 For this repository, the most reliable first-run path is the preset-backed launcher:

agentbyte-0.33.0/pyproject.toml

@@ -0,0 +1,112 @@
+[project]
+name = "agentbyte"
+dynamic = ["version"]
+description = "A toolkit for designing multiagent systems"
+readme = "README.md"
+license = "LicenseRef-Proprietary"
+authors = [
+    { name = "MrDataPsycho", email = "mr.data.psycho@gmail.com" }
+]
+requires-python = ">=3.11"
+dependencies = [
+    "httpx>=0.28.1",
+    "pydantic>=2.12.5",
+    "pydantic-settings>=2.13.0",
+    "pyyaml>=6.0.3",
+]
+
+[project.optional-dependencies]
+# Published feature extras: `pip install "agentbyte[openai,sqlite]"`.
+openai = [
+    "openai>=1.107.1",
+]
+azureopenai = [
+    "openai>=1.107.1",
+    "azure-identity>=1.25.1",
+]
+aws = [
+    "aioboto3>=15.5.0",
+]
+otel = [
+    "opentelemetry-api>=1.39.1",
+    "opentelemetry-exporter-otlp-proto-http>=1.39.1",
+    "opentelemetry-sdk>=1.39.1",
+]
+webui = [
+    "fastapi>=0.135.2",
+    "uvicorn[standard]>=0.44.0",
+]
+sqlite = [
+    "aiosqlite>=0.22.1",
+]
+postgres = [
+    "sqlalchemy[asyncio]>=2.0.43",
+    "sqlmodel>=0.0.38",
+    "asyncpg>=0.31.0",
+]
+sql = [
+    "agentbyte[sqlite,postgres]",
+]
+tiktoken = [
+    "tiktoken>=0.9.0",
+]
+optim = [
+    "gepa>=0.1",
+]
+viz = [
+    "graphviz>=0.21",
+]
+research = [
+    "beautifulsoup4>=4.12",
+    "html2text>=2024.2",
+    "arxiv>=2.1",
+    "youtube-transcript-api>=0.6",
+]
+all = [
+    "agentbyte[openai,azureopenai,aws,otel,webui,sql,tiktoken,optim,viz,research]",
+]
+
+
+[project.scripts]
+agentbyte = "agentbyte.cli:main"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.version]
+path = "src/agentbyte/__about__.py"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/agentbyte"]
+
+[tool.hatch.build.targets.sdist]
+include = ["/src", "/tests", "/README.md", "/LICENSE", "/CHANGELOG.md", "/src/agentbyte/webui/ui"]
+
+[dependency-groups]
+# Local-only tooling, never published. Contributors: `uv sync --all-extras --all-groups`.
+test = [
+    "pytest>=9.0.1",
+    "pytest-asyncio>=1.3.0",
+    "pytest-cov>=7.0.0",
+    "ruff>=0.15.0",
+]
+dev = [
+    "fastexcel>=0.19.0",
+    "ipykernel>=7.1.0",
+    "nbclient>=0.10.4",
+    "nbformat>=5.10.4",
+    "polars>=1.40.0",
+    "pymupdf-layout>=1.26.6",
+    "pymupdf4llm>=0.2.9",
+]
+experiment = [
+    "tiktoken>=0.9.0",
+    "yake>=0.4.8",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+norecursedirs = ["external_lib"]
+# Dump every thread's stack if a test stalls, instead of hanging silently.
+faulthandler_timeout = 60

agentbyte-0.33.0/src/agentbyte/__about__.py

@@ -0,0 +1,2 @@
+__version__ = "0.33.0"
+VERSION = __version__

{agentbyte-0.32.2 → agentbyte-0.33.0}/src/agentbyte/__init__.py

@@ -14,6 +14,7 @@ Architecture Layers:
 """
 
 from agentbyte.__about__ import VERSION
+from agentbyte._optional import MissingExtraError
 from agentbyte.agents import Agent
 from agentbyte.context_providers import (
     ContextProvider,
@@ -129,6 +130,7 @@ except Exception:
 
 __all__ = [
     "VERSION",
+    "MissingExtraError",
     "Agent",
     "Entity",
     "configure_notebook_logging",

agentbyte-0.33.0/src/agentbyte/_optional.py

@@ -0,0 +1,39 @@
+"""One way to import optional dependencies and report the extra that provides them."""
+
+from __future__ import annotations
+
+import importlib
+from types import ModuleType
+
+
+class MissingExtraError(ModuleNotFoundError):
+    """An optional dependency is not installed; the message names the extra to install."""
+
+    def __init__(self, feature: str, extra: str, *, name: str | None = None) -> None:
+        super().__init__(
+            f"{feature} needs the '{extra}' extra: pip install 'agentbyte[{extra}]' "
+            f"(in this repository: uv sync --extra {extra})",
+            name=name,
+        )
+        self.feature = feature
+        self.extra = extra
+
+
+def require(module: str, *, extra: str, feature: str) -> ModuleType:
+    """Import ``module`` or raise ``MissingExtraError`` naming ``extra``.
+
+    Only a missing ``module`` (or one of its parent packages) is reported as a
+    missing extra; an installed package that fails on its own imports re-raises.
+    """
+    try:
+        return importlib.import_module(module)
+    except ModuleNotFoundError as exc:
+        if not is_missing(exc, module):
+            raise
+        raise MissingExtraError(feature, extra, name=exc.name) from exc
+
+
+def is_missing(exc: ModuleNotFoundError, module: str) -> bool:
+    """Whether ``exc`` means ``module`` itself (or a parent package) is not installed."""
+    missing = exc.name or ""
+    return bool(missing) and (module == missing or module.startswith(f"{missing}."))

agentbyte-0.33.0/src/agentbyte/cli/__main__.py

@@ -0,0 +1,6 @@
+"""Allow ``python -m agentbyte.cli``."""
+
+from agentbyte.cli.main import main
+
+if __name__ == "__main__":
+    main()