dataenginex 0.10.0__tar.gz → 1.0.0__tar.gz

{dataenginex-0.10.0 → dataenginex-1.0.0}/.github/workflows/ci.yml

@@ -7,6 +7,10 @@ on:
     branches: [main, dev]
   workflow_dispatch:
 
+  schedule:
+    # Weekly Python version compatibility check
+    - cron: '0 0 * * 0'
+
 permissions:
   contents: read
 
@@ -26,7 +30,7 @@ jobs:
       - run: uv run poe quality
 
   test:
-    name: Tests
+    name: Tests (Python 3.13)
     runs-on: ubuntu-latest
     needs: quality
     steps:
@@ -45,3 +49,21 @@ jobs:
           fail_ci_if_error: false
         env:
           CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
+
+  test-compat:
+    name: Python Compatibility
+    runs-on: ubuntu-latest
+    if: github.event_name == 'schedule'
+    strategy:
+      matrix:
+        python-version: ["3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v6
+      - uses: astral-sh/setup-uv@v7
+        with:
+          version: "latest"
+          python-version: ${{ matrix.python-version }}
+      - run: uv sync --group ml
+        env:
+          UV_PROJECT_ENVIRONMENT: .venv
+      - run: uv run poe check-all

{dataenginex-0.10.0 → dataenginex-1.0.0}/.github/workflows/docs-notify.yml

@@ -9,7 +9,7 @@ jobs:
   notify:
     runs-on: ubuntu-latest
     steps:
-      - uses: peter-evans/repository-dispatch@v3
+      - uses: peter-evans/repository-dispatch@v4
         with:
           token: ${{ secrets.DOCS_DISPATCH_TOKEN }}
           repository: TheDataEngineX/docs

{dataenginex-0.10.0 → dataenginex-1.0.0}/CHANGELOG.md

@@ -28,6 +28,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.0.0] - 2026-04-07
+
+### Highlights
+
+- **Complete Data + ML + AI Framework**: All phases from the v1.0 master plan implemented — config-driven pipeline via `dex.yaml`, BackendRegistry pattern for swappable backends, unified CLI.
+- **Data Layer**: DuckDB connector (default), CSV connector, PipelineRunner with DAG resolution, transforms (filter, derive, cast, deduplicate), quality gates (completeness, uniqueness), column-level lineage tracking, built-in cron scheduler.
+- **ML Layer**: SQLite-backed experiment tracker, DuckDB feature store, sklearn/xgboost training integration, model registry with versioning (dev → staging → production), built-in model serving via FastAPI, PSI drift detection.
+- **AI Layer**: Built-in ReAct agent runtime, Ollama LLM provider (default), tool registry (sql_query, predict, search), BM25 sparse retrieval (DuckDB FTS), dense vector retrieval (DuckDB VSS HNSW), hybrid retrieval with RRF fusion, agent memory (short-term + episodic).
+- **CLI Commands**: `dex init`, `dex validate`, `dex version`, `dex serve`, `dex run`, `dex train`, `dex agent`, `dex query`.
+- **API**: FastAPI app factory, JWT auth, rate limiting, health endpoints, project CRUD, pipeline run/status, data explorer, ML experiments/models, agent chat/manage, WebSocket for live logs and streaming.
+- **Backend Registry Pattern**: Every subsystem follows ABC + BackendRegistry[T] pattern with built-in implementations and optional extras (Dagster, MLflow, Qdrant, LanceDB, sentence-transformers, PySpark).
+
+### Breaking Changes
+
+- **FastAPI now optional**: Core install (`pip install dataenginex`) includes only lightweight deps. Install `[api]` extra for FastAPI/uvicorn: `pip install dataenginex[api]`
+- **Cloud SDKs now optional**: Core install no longer requires boto3/google-cloud-storage/google-cloud-bigquery. Install `[cloud]` extra: `pip install dataenginex[cloud]`
+- **Routers moved**: API routers moved to application packages. Use `from dataenginex.api import ...` directly (requires `[api]` extra)
+- **Root `__init__.py` slimmed**: Re-exports removed. Import from submodules directly: `from dataenginex.api import HealthChecker` etc.
+
+### Added
+
+- **Full project templates**: `dex init --template [minimal|data-pipeline|ml-project|ai-agent|full-stack|career-intelligence]`
+- **Docker support**: Multi-stage Dockerfile (`ghcr.io/thedataenginex/dex`), docker-compose.yml for full stack
+- **SecOps**: PII scanning in pipelines, masking, audit trail
+- **Quality schema**: Spark audit integration for data quality validation
+- **Examples**: 5 runnable examples in `examples/` directory
+
+### Verification checklist
+
+1. `uv run poe lint` — Ruff checks clean
+2. `uv run poe typecheck` — mypy strict (all modules)
+3. `uv run poe test` — 663 passed, 36 skipped
+4. `pip install dataenginex` — installs successfully
+5. `dex validate dex.yaml` — validates config
+6. `dex version` — shows version
+
+[Unreleased]: https://github.com/TheDataEngineX/DEX/compare/v1.0.0...HEAD
+[1.0.0]: https://github.com/TheDataEngineX/DEX/releases/tag/v1.0.0
+
 ## [0.7.1] - 2026-03-17
 
 ### Fixed

{dataenginex-0.10.0 → dataenginex-1.0.0}/CLAUDE.md

@@ -11,7 +11,7 @@ Goal is to save Claude code tokens for lower cost without loosing quality.
 
 | Package | Location | Purpose |
 |---------|----------|---------|
-| `dataenginex` | `src/dataenginex/` | Core framework — config system, backend registry, CLI, API, ML, AI agents |
+| `dataenginex` | `src/dataenginex/` | Core framework — config, registry, CLI, API, ML, AI (routing, runtime, memory, observability, workflows) |
 
 **Stack:** Python 3.13+ · FastAPI · DuckDB · structlog · Pydantic · Click · Rich · uv · Ruff · mypy strict · pytest
 

dataenginex-1.0.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,57 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment:
+
+- Using welcoming and inclusive language
+- Being respectful of differing viewpoints and experiences
+- Gracefully accepting constructive criticism
+- Focusing on what is best for the community
+- Showing empathy towards other community members
+
+Examples of unacceptable behavior:
+
+- The use of sexualized language or imagery, and sexual attention or advances
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information without explicit permission
+- Other conduct which could reasonably be considered inappropriate
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+**conduct@thedataenginex.dev**.
+
+All complaints will be reviewed and investigated promptly and fairly.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org),
+version 2.1, available at
+<https://www.contributor-covenant.org/version/2/1/code_of_conduct.html>.

{dataenginex-0.10.0 → dataenginex-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dataenginex
-Version: 0.10.0
+Version: 1.0.0
 Summary: DataEngineX - Core framework for AI/ML/Data engineering projects
 Author-email: Jay <jayapal.myaka99@gmail.com>
 License: MIT License
@@ -40,7 +40,6 @@ Requires-Dist: prometheus-client>=0.24.1
 Requires-Dist: pyarrow>=23.0.1
 Requires-Dist: pydantic>=2.10.0
 Requires-Dist: python-dotenv>=1.2.1
-Requires-Dist: python-json-logger>=4.0.0
 Requires-Dist: pyyaml>=6.0.3
 Requires-Dist: rich>=14.3.3
 Requires-Dist: structlog>=25.5.0

{dataenginex-0.10.0 → dataenginex-1.0.0}/README.md

@@ -112,7 +112,7 @@ ______________________________________________________________________
 
 ## Development
 
-See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for full setup.
+See [docs/development.md](docs/development.md) for full setup.
 
 ```bash
 uv run poe check-all         # lint + typecheck + tests

dataenginex-1.0.0/SECURITY.md

@@ -0,0 +1,71 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported |
+|---------|-----------|
+| Latest minor release (1.0.x) | ✅ |
+| Previous minor release | ✅ (security fixes only) |
+| Older versions | ❌ |
+
+## Reporting a Vulnerability
+
+**Do NOT open a public issue for security vulnerabilities.**
+
+Instead, please report them via one of these channels:
+
+1. **Email**: security@thedataenginex.dev
+2. **GitHub Security Advisories**: Use the "Report a vulnerability" button on the Security tab
+
+### What to Include
+
+- Description of the vulnerability
+- Steps to reproduce
+- Potential impact assessment
+- Suggested fix (if any)
+
+### Response Timeline
+
+| Stage | Timeline |
+|-------|----------|
+| Acknowledgment | Within 48 hours |
+| Initial assessment | Within 5 business days |
+| Fix development | Within 30 days (critical), 90 days (non-critical) |
+| Public disclosure | After fix is released |
+
+## Disclosure Policy
+
+We follow [coordinated disclosure](https://en.wikipedia.org/wiki/Coordinated_vulnerability_disclosure).
+We will credit reporters in the security advisory unless they prefer to remain anonymous.
+
+## Security Practices
+
+DataEngineX follows these security practices:
+
+- **No hardcoded secrets** — all credentials via environment variables
+- **Parameterized queries** — never SQL concatenation
+- **Input validation** — Pydantic models at API boundaries
+- **Dependency auditing** — automated via `uv run poe security`
+- **Pickle safety** — SafeUnpickler with HMAC verification for model loading
+- **Container security** — non-root users, minimal base images
+- **HTTPS only** — all production traffic encrypted
+- **Least privilege** — minimal permissions for service accounts
+
+## Security-Related Dependencies
+
+| Dependency | Purpose | Security Note |
+|------------|---------|---------------|
+| pydantic | Config validation | Validates all inputs |
+| python-dotenv | Env var loading | Never commit .env files |
+| httpx | HTTP client | Timeout configured |
+| structlog | Logging | No PII in logs by default |
+
+## Auditing
+
+Run security audits locally:
+
+```bash
+uv run poe security  # pip-audit for vulnerabilities
+```
+
+CI runs `pip-audit` and dependency scanning on every PR.

{dataenginex-0.10.0 → dataenginex-1.0.0}/docs/ci-cd.md

@@ -407,4 +407,4 @@ gh workflow run pypi-publish.yml -f tag=v<version>
 
 ______________________________________________________________________
 
-**[← Back to Documentation Hub](docs-hub.md)**
+**[← Back to Documentation](index.md)**

{dataenginex-0.10.0 → dataenginex-1.0.0}/docs/index.md

@@ -4,7 +4,7 @@ Welcome to the DEX docs site.
 
 Start here:
 
-- [Documentation Hub](docs-hub.md)
 - [Development Setup](development.md)
 - [Contributing](contributing.md)
 - [Architecture](architecture.md)
+- [Quickstart](quickstart.md)

{dataenginex-0.10.0 → dataenginex-1.0.0}/docs/observability.md

@@ -795,4 +795,4 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
-**[← Back to Documentation Hub](docs-hub.md)**
+**[← Back to Documentation](index.md)**

{dataenginex-0.10.0 → dataenginex-1.0.0}/docs/sdlc.md

@@ -203,4 +203,4 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
-**[← Back to Documentation Hub](docs-hub.md)**
+**[← Back to Documentation](index.md)**

{dataenginex-0.10.0 → dataenginex-1.0.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "dataenginex"
-version = "0.10.0"
+version = "1.0.0"
 description = "DataEngineX - Core framework for AI/ML/Data engineering projects"
 authors = [
     {name = "Jay", email = "jayapal.myaka99@gmail.com"}
@@ -22,7 +22,7 @@ dependencies = [
     "fastapi>=0.135.1",
     "uvicorn>=0.42.0",
     "structlog>=25.5.0",
-    "python-json-logger>=4.0.0",
+    # OpenTelemetry
     "opentelemetry-api>=1.40.0",
     "opentelemetry-sdk>=1.40.0",
     "opentelemetry-instrumentation-fastapi>=0.61b0",

dataenginex-1.0.0/src/dataenginex/ai/__init__.py

@@ -0,0 +1,79 @@
+"""AI layer — agents, retrieval, tools, routing, runtime, memory, observability, workflows.
+
+Public API::
+
+    from dataenginex.ai import (
+        retriever_registry, agent_registry, tool_registry,
+        BuiltinRetriever, BuiltinAgentRuntime,
+        ModelRouter, BaseProvider,
+        Sandbox, SandboxConfig,
+        AuditLog, CostTracker,
+        AgentDAG, Condition,
+        ShortTermMemory, LongTermMemory, EpisodicMemory,
+        CheckpointManager, AgentExecutor, AgentConfig,
+        AgentMetrics,
+    )
+"""
+
+from __future__ import annotations
+
+from dataenginex.ai.agents import agent_registry
+from dataenginex.ai.agents.builtin import BuiltinAgentRuntime
+from dataenginex.ai.memory.base import BaseMemory, MemoryEntry, ShortTermMemory
+from dataenginex.ai.memory.episodic import Episode, EpisodicMemory
+from dataenginex.ai.memory.long_term import LongTermMemory
+from dataenginex.ai.observability.audit import AuditEntry, AuditLog
+from dataenginex.ai.observability.cost import CostTracker, TokenUsage
+from dataenginex.ai.observability.metrics import AgentMetrics
+from dataenginex.ai.retrieval import retriever_registry
+from dataenginex.ai.retrieval.builtin import BuiltinRetriever
+from dataenginex.ai.routing.router import BaseProvider, ModelRouter
+from dataenginex.ai.runtime.checkpoint import Checkpoint, CheckpointManager
+from dataenginex.ai.runtime.executor import AgentConfig, AgentExecutor, AgentResponse
+from dataenginex.ai.runtime.sandbox import Sandbox, SandboxConfig, SandboxResult
+from dataenginex.ai.tools import ToolRegistry, ToolSpec, tool_registry
+from dataenginex.ai.workflows.conditions import Condition
+from dataenginex.ai.workflows.dag import AgentDAG
+from dataenginex.ai.workflows.human_loop import ApprovalGate
+
+__all__ = [
+    # Registries
+    "agent_registry",
+    "retriever_registry",
+    "tool_registry",
+    # Agents
+    "BuiltinAgentRuntime",
+    "BuiltinRetriever",
+    # Tools
+    "ToolRegistry",
+    "ToolSpec",
+    # Memory
+    "BaseMemory",
+    "MemoryEntry",
+    "ShortTermMemory",
+    "LongTermMemory",
+    "EpisodicMemory",
+    "Episode",
+    # Observability
+    "AuditEntry",
+    "AuditLog",
+    "CostTracker",
+    "TokenUsage",
+    "AgentMetrics",
+    # Routing
+    "BaseProvider",
+    "ModelRouter",
+    # Runtime
+    "AgentConfig",
+    "AgentExecutor",
+    "AgentResponse",
+    "Checkpoint",
+    "CheckpointManager",
+    "Sandbox",
+    "SandboxConfig",
+    "SandboxResult",
+    # Workflows
+    "AgentDAG",
+    "ApprovalGate",
+    "Condition",
+]

dataenginex-1.0.0/src/dataenginex/ai/memory/__init__.py

@@ -0,0 +1,16 @@
+"""Agent memory — short-term, long-term, and episodic memory."""
+
+from __future__ import annotations
+
+from dataenginex.ai.memory.base import BaseMemory, MemoryEntry, ShortTermMemory
+from dataenginex.ai.memory.episodic import Episode, EpisodicMemory
+from dataenginex.ai.memory.long_term import LongTermMemory
+
+__all__ = [
+    "BaseMemory",
+    "Episode",
+    "EpisodicMemory",
+    "LongTermMemory",
+    "MemoryEntry",
+    "ShortTermMemory",
+]

dataenginex-1.0.0/src/dataenginex/ai/memory/base.py

@@ -0,0 +1,59 @@
+"""Agent memory system — short-term, long-term, and episodic memory."""
+
+from __future__ import annotations
+
+from abc import ABC, abstractmethod
+from dataclasses import dataclass, field
+from typing import Any
+
+
+@dataclass
+class MemoryEntry:
+    """A single memory entry."""
+
+    content: str
+    role: str  # "user", "assistant", "system", "tool"
+    metadata: dict[str, Any] = field(default_factory=dict)
+    timestamp: float = 0.0
+
+
+class BaseMemory(ABC):
+    """Abstract base class for agent memory."""
+
+    @abstractmethod
+    def add(self, entry: MemoryEntry) -> None:
+        """Add a memory entry."""
+
+    @abstractmethod
+    def search(self, query: str, top_k: int = 5) -> list[MemoryEntry]:
+        """Search memory by semantic similarity."""
+
+    @abstractmethod
+    def recent(self, n: int = 10) -> list[MemoryEntry]:
+        """Get the most recent entries."""
+
+    @abstractmethod
+    def clear(self) -> None:
+        """Clear all memory."""
+
+
+class ShortTermMemory(BaseMemory):
+    """Session/conversation memory — lives in-process, lost on restart."""
+
+    def __init__(self, max_entries: int = 100) -> None:
+        self._entries: list[MemoryEntry] = []
+        self._max = max_entries
+
+    def add(self, entry: MemoryEntry) -> None:
+        self._entries.append(entry)
+        if len(self._entries) > self._max:
+            self._entries.pop(0)
+
+    def search(self, query: str, top_k: int = 5) -> list[MemoryEntry]:
+        return [e for e in self._entries if query.lower() in e.content.lower()][:top_k]
+
+    def recent(self, n: int = 10) -> list[MemoryEntry]:
+        return self._entries[-n:]
+
+    def clear(self) -> None:
+        self._entries.clear()

dataenginex-1.0.0/src/dataenginex/ai/memory/episodic.py

@@ -0,0 +1,40 @@
+"""Episodic memory — experience replay for task-based learning."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from pydantic import BaseModel
+
+
+class Episode(BaseModel):
+    """A recorded agent episode — a full task execution with outcome."""
+
+    task: str
+    steps: list[dict[str, Any]]
+    outcome: str
+    reward: float
+    timestamp: float
+
+
+class EpisodicMemory:
+    """Experience replay memory — stores and retrieves past episodes."""
+
+    def __init__(self) -> None:
+        self._episodes: list[Episode] = []
+
+    def add_episode(self, episode: Episode) -> None:
+        self._episodes.append(episode)
+
+    def recall_similar(self, task: str, top_k: int = 5) -> list[Episode]:
+        task_lower = task.lower()
+        scored: list[tuple[int, Episode]] = []
+        for ep in self._episodes:
+            score = sum(1 for word in task_lower.split() if word in ep.task.lower())
+            if score > 0:
+                scored.append((score, ep))
+        scored.sort(key=lambda x: x[0], reverse=True)
+        return [ep for _, ep in scored[:top_k]]
+
+    def best_episodes(self, top_k: int = 5) -> list[Episode]:
+        return sorted(self._episodes, key=lambda e: e.reward, reverse=True)[:top_k]

dataenginex-1.0.0/src/dataenginex/ai/memory/long_term.py

@@ -0,0 +1,52 @@
+"""Long-term memory — keyword-searchable persistent memory store."""
+
+from __future__ import annotations
+
+import json
+import time
+from dataclasses import asdict
+from pathlib import Path
+
+from dataenginex.ai.memory.base import BaseMemory, MemoryEntry
+
+
+class LongTermMemory(BaseMemory):
+    """Persistent memory with keyword search — no external vector DB required.
+
+    Data is stored as a flat list and scored by keyword overlap.
+    Call :meth:`persist` to write to disk and :meth:`load_from_file` to restore.
+    """
+
+    def __init__(self) -> None:
+        self._entries: list[MemoryEntry] = []
+
+    def add(self, entry: MemoryEntry) -> None:
+        if not entry.timestamp:
+            entry.timestamp = time.time()
+        self._entries.append(entry)
+
+    def search(self, query: str, top_k: int = 5) -> list[MemoryEntry]:
+        query_lower = query.lower()
+        scored: list[tuple[int, MemoryEntry]] = []
+        for entry in self._entries:
+            score = sum(1 for word in query_lower.split() if word in entry.content.lower())
+            if score > 0:
+                scored.append((score, entry))
+        scored.sort(key=lambda x: x[0], reverse=True)
+        return [e for _, e in scored[:top_k]]
+
+    def recent(self, n: int = 10) -> list[MemoryEntry]:
+        return self._entries[-n:]
+
+    def clear(self) -> None:
+        self._entries.clear()
+
+    def persist(self, path: str) -> None:
+        """Persist all memory entries to a JSON file at *path*."""
+        data = [asdict(e) for e in self._entries]
+        Path(path).write_text(json.dumps(data, indent=2), encoding="utf-8")
+
+    def load_from_file(self, path: str) -> None:
+        """Replace in-memory entries with those from a JSON file at *path*."""
+        raw: list[dict[str, object]] = json.loads(Path(path).read_text(encoding="utf-8"))
+        self._entries = [MemoryEntry(**item) for item in raw]  # type: ignore[arg-type]

dataenginex-1.0.0/src/dataenginex/ai/observability/__init__.py

@@ -0,0 +1,9 @@
+"""Observability — audit logging, cost tracking, and metrics."""
+
+from __future__ import annotations
+
+from dataenginex.ai.observability.audit import AuditEntry, AuditLog
+from dataenginex.ai.observability.cost import CostTracker, TokenUsage
+from dataenginex.ai.observability.metrics import AgentMetrics
+
+__all__ = ["AgentMetrics", "AuditEntry", "AuditLog", "CostTracker", "TokenUsage"]

dataenginex-1.0.0/src/dataenginex/ai/observability/audit.py

@@ -0,0 +1,34 @@
+"""Audit logging — track every agent action for compliance and debugging."""
+
+from __future__ import annotations
+
+from pydantic import BaseModel
+
+
+class AuditEntry(BaseModel):
+    """A single audit log entry."""
+
+    agent_name: str
+    action: str
+    input: str
+    output: str
+    timestamp: float
+    reasoning: str = ""
+
+
+class AuditLog:
+    """In-memory audit log for agent actions."""
+
+    def __init__(self) -> None:
+        self._entries: list[AuditEntry] = []
+
+    def log(self, entry: AuditEntry) -> None:
+        """Record an audit entry."""
+        self._entries.append(entry)
+
+    def get_entries(self, agent_name: str | None = None, limit: int = 100) -> list[AuditEntry]:
+        """Get audit entries, optionally filtered by agent name."""
+        if agent_name is None:
+            return self._entries[-limit:]
+        filtered = [e for e in self._entries if e.agent_name == agent_name]
+        return filtered[-limit:]

dataenginex-1.0.0/src/dataenginex/ai/observability/cost.py

@@ -0,0 +1,57 @@
+"""Token usage and cost tracking for LLM calls."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from pydantic import BaseModel
+
+
+class TokenUsage(BaseModel):
+    """Token usage for a single LLM call."""
+
+    model: str
+    tokens_in: int
+    tokens_out: int
+    cost_usd: float
+    agent_name: str = ""
+
+
+class CostTracker:
+    """Tracks cumulative token usage and costs across agents."""
+
+    def __init__(self) -> None:
+        self._records: list[TokenUsage] = []
+
+    def record(self, usage: TokenUsage) -> None:
+        """Record a token usage entry."""
+        self._records.append(usage)
+
+    def total_cost(self, agent_name: str | None = None) -> float:
+        """Get total cost in USD, optionally filtered by agent."""
+        if agent_name is None:
+            return sum(r.cost_usd for r in self._records)
+        return sum(r.cost_usd for r in self._records if r.agent_name == agent_name)
+
+    def summary(self) -> dict[str, Any]:
+        """Get a summary of all token usage and costs."""
+        total_in = sum(r.tokens_in for r in self._records)
+        total_out = sum(r.tokens_out for r in self._records)
+        return {
+            "total_records": len(self._records),
+            "total_tokens_in": total_in,
+            "total_tokens_out": total_out,
+            "total_cost_usd": self.total_cost(),
+            "by_model": self._by_model(),
+        }
+
+    def _by_model(self) -> dict[str, dict[str, Any]]:
+        """Break down costs by model."""
+        result: dict[str, dict[str, Any]] = {}
+        for r in self._records:
+            if r.model not in result:
+                result[r.model] = {"tokens_in": 0, "tokens_out": 0, "cost_usd": 0.0}
+            result[r.model]["tokens_in"] += r.tokens_in
+            result[r.model]["tokens_out"] += r.tokens_out
+            result[r.model]["cost_usd"] += r.cost_usd
+        return result