agentexec 0.1.2__tar.gz → 0.1.4__tar.gz

agentexec-0.1.4/.claude/skills/prepare-release/SKILL.md

@@ -0,0 +1,57 @@
+---
+name: prepare-release
+description: Prepare a new release by updating CHANGELOG.md with changes since the last tag and bumping version numbers in pyproject.toml and ui/package.json
+---
+
+# Prepare Release
+
+When preparing a release, follow these steps:
+
+## 1. Fetch tags and identify the last release
+
+```bash
+git fetch --tags
+git tag --sort=-creatordate | head -5
+```
+
+## 2. Get commits since the last release
+
+```bash
+git log --oneline <last-tag>..HEAD
+```
+
+## 3. Read current files
+
+Read these files to understand current state:
+- `CHANGELOG.md` - to understand the format and existing entries
+- `pyproject.toml` - to get current version
+- `ui/package.json` - to get current UI version
+
+## 4. Update CHANGELOG.md
+
+- Add a new version section at the top (below the `# Changelog` header)
+- Organize changes into categories:
+  - **Breaking Changes** - API changes that require user action
+  - **New Features** - New functionality
+  - **Internal Improvements** - Refactoring, tests, tooling
+- Use bold headers for each change group
+- Use bullet points for details
+- If there was an "Unreleased" section, rename it to the previous version
+
+## 5. Bump versions
+
+Update the version string in:
+- `pyproject.toml` - the `version` field under `[project]`
+- `ui/package.json` - the `"version"` field
+
+Use semantic versioning:
+- Patch (0.0.x) for bug fixes
+- Minor (0.x.0) for new features
+- Major (x.0.0) for breaking changes
+
+## 6. Summary
+
+After completing, summarize what was updated:
+- New version number
+- Key changes added to changelog
+- Files modified

agentexec-0.1.4/.github/workflows/docker-publish.yml

@@ -0,0 +1,65 @@
+name: Publish Docker Image
+
+on:
+  release:
+    types: [published]
+  # Allow manual trigger for testing
+  workflow_dispatch:
+    inputs:
+      tag:
+        description: 'Image tag (defaults to "dev")'
+        required: false
+        default: 'dev'
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: agent-ci/agentexec-worker
+
+permissions:
+  contents: read
+  packages: write
+
+jobs:
+  build-and-push:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Log in to Container Registry
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata for Docker
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            # Tag with version on release (e.g., v0.1.2 -> 0.1.2)
+            type=semver,pattern={{version}}
+            # Tag with major.minor on release (e.g., v0.1.2 -> 0.1)
+            type=semver,pattern={{major}}.{{minor}}
+            # Tag as "latest" on release
+            type=raw,value=latest,enable=${{ github.event_name == 'release' }}
+            # Tag with input value on manual dispatch
+            type=raw,value=${{ inputs.tag }},enable=${{ github.event_name == 'workflow_dispatch' }}
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v5
+        with:
+          context: ./docker
+          file: ./docker/Dockerfile
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+          platforms: linux/amd64,linux/arm64

agentexec-0.1.4/.github/workflows/npm-publish.yml

@@ -0,0 +1,46 @@
+name: Publish agentexec-ui to npm
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Version to publish (leave empty to use package.json version)'
+        required: false
+        type: string
+
+permissions:
+  contents: read
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+
+    defaults:
+      run:
+        working-directory: ui
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Setup Node.js
+      uses: actions/setup-node@v4
+      with:
+        node-version: '20'
+        registry-url: 'https://registry.npmjs.org'
+
+    - name: Install dependencies
+      run: npm install
+
+    - name: Update version (if specified)
+      if: inputs.version != ''
+      run: npm version ${{ inputs.version }} --no-git-tag-version
+
+    - name: Build package
+      run: npm run build
+
+    - name: Publish to npm
+      run: npm publish --access public
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

agentexec-0.1.4/CHANGELOG.md

@@ -0,0 +1,211 @@
+# Changelog
+
+## v0.1.4
+
+### Breaking Changes
+
+**Renamed `WorkerPool` to `Pool`**
+- `ax.WorkerPool` is now `ax.Pool` for cleaner API
+- Update imports: `from agentexec import Pool`
+
+**Activity percentage field renamed**
+- `completion_%` renamed to `percentage` for cleaner field naming
+
+### New Features
+
+**Pipelines run on workers**
+- Pipelines can now be executed on worker processes
+- Register pipelines with the pool and enqueue them like tasks
+
+**Tracker for stateful counters**
+- New `Tracker` class for managing stateful counters across workers
+- Useful for tracking progress, metrics, and distributed state
+
+**Strict Pipeline type flow validation**
+- All step parameters and return types must be `BaseModel` subclasses
+- Type flow between consecutive steps is validated at runtime
+- Tuple returns are unpacked and matched to next step's parameters
+- Final step must return a single `BaseModel` (not a tuple)
+- Empty pipelines raise `RuntimeError` at class definition time
+
+### Internal Improvements
+
+**Type checking with `ty`**
+- Added `ty` type checker to development workflow
+- Better Protocol definitions for step handlers
+- Improved type hints throughout pipeline module
+
+**Better Pipeline flow tests**
+- Comprehensive test coverage for valid and invalid type flows
+- Tests for tuple unpacking, subclass compatibility, count mismatches
+- Tests for primitive type rejection and edge cases
+
+## v0.1.3
+
+### Breaking Changes
+
+**Self-describing JSON serialization replaces pickle**
+- Task results now use JSON serialization with embedded type information (similar to pickle)
+- Automatically stores fully qualified class name with data for type reconstruction
+- No longer requires `TaskDefinition` registry for result deserialization
+- `ax.gather()` now works with tasks created via `ax.enqueue()` without pool context
+- **Migration**: Clear Redis or wait for TTL expiry on old pickled results
+
+**TaskHandler Protocol enforces BaseModel returns**
+- Task handlers must return a Pydantic `BaseModel` instance (not `None` or arbitrary objects)
+- Return type is automatically inferred and validated at registration time
+- Enables type-safe result retrieval and automatic serialization
+
+### New Features
+
+**State backend abstraction**
+- Introduced `StateBackend` Protocol for pluggable state storage implementations
+- Current Redis implementation moved to `agentexec.state.redis_backend`
+- Backend modules verified against protocol at import time via `cast()`
+- Prepares foundation for alternative backends (in-memory, DynamoDB, etc.)
+
+**Improved async patterns**
+- `brpop()` is now a proper async function (was sync returning coroutine)
+- Consistent async/await usage across state operations
+- Better type hints and IDE support
+
+**Enhanced type safety**
+- `TaskHandler` Protocol with support for both sync and async handlers
+- Proper type annotations for all state backend operations
+- `serialize()` and `deserialize()` type-enforced for `BaseModel` only
+
+### Documentation
+
+**Comprehensive documentation added**
+- API reference for core modules (activity, pipeline, runner, task)
+- Conceptual guides (architecture, task lifecycle, worker pool)
+- Deployment guides (Docker, production best practices)
+- Usage guides (basic usage, pipelines, FastAPI integration, OpenAI runner)
+- Getting started (installation, quickstart, configuration)
+- Contributing guide
+
+### UI & Tooling
+
+**React frontend and component library**
+- Added `agentexec-ui` npm package with reusable React components
+- Pre-built UI for agent monitoring and activity tracking
+- TanStack Query integration for real-time updates
+- React Router for navigation between agent list and detail views
+
+**Docker deployment**
+- Docker worker image for containerized deployments
+- GitHub Actions for automated Docker image publishing to GitHub Container Registry
+- GitHub Actions for automated npm publishing of UI components
+
+### Testing
+
+**Comprehensive test coverage**
+- Achieved 89% code coverage
+- Added unit tests for all core modules:
+  - State backend and serialization (`test_state.py`, `test_state_backend.py`)
+  - Self-describing results (`test_self_describing_results.py`)
+  - Activity tracking schemas (`test_activity_schemas.py`)
+  - Pipeline orchestration (`test_pipeline.py`)
+  - Task queue operations (`test_queue.py`)
+  - Worker events and logging (`test_worker_event.py`, `test_worker_logging.py`)
+  - Database operations (`test_db.py`)
+  - Configuration (`test_config.py`)
+
+### Internal Improvements
+
+**Redis client refactoring**
+- Removed `core/redis_client.py` in favor of state backend abstraction
+- Lazy connection initialization for both async and sync Redis clients
+- Proper connection cleanup in `backend.close()`
+
+**Key formatting consistency**
+- All state keys use consistent `agentexec:` prefix via `backend.format_key()`
+- Results: `agentexec:result:{agent_id}`
+- Events: `agentexec:event:{name}:{id}`
+- Logs channel: `agentexec:logs`
+
+**Standardized function signatures**
+- `get_result()` and `gather()` return `BaseModel` directly (not JSON strings)
+- Consistent parameter ordering across state module functions
+- Better docstrings with type information
+
+## v0.1.2
+
+### New Features
+
+**Pipelines**
+- Multi-step workflow orchestration with `ax.Pipeline`
+- Define steps with `@pipeline.step(order)` decorator
+- Parallel task execution with `ax.gather()`
+- Result retrieval with `ax.get_result()`
+
+**Worker logging via Redis pubsub**
+- Workers publish logs to Redis, collected by main process
+- Use `pool.run()` to see worker logs in real-time
+
+### Internal Improvements
+
+**Reorganized worker module**
+- Worker code moved to `agentexec.worker` subpackage
+- `RedisEvent` for cross-process shutdown coordination
+- `get_worker_logger()` configures logging and returns logger in one call
+
+**Refactored Redis client usage**
+- Added `get_redis_sync()` for synchronous Redis operations
+- Sync/async Redis clients for different contexts
+
+## v0.1.1
+
+### Breaking Changes
+
+**Async `enqueue()` function**
+- `ax.enqueue()` is now async and must be awaited:
+  ```python
+  task = await ax.enqueue("task_name", MyContext(key="value"))
+  ```
+
+**Type-safe context with Pydantic BaseModel**
+- Task context must be a Pydantic `BaseModel` instead of a raw `dict`
+- Context class is automatically inferred from handler type hints:
+  ```python
+  class ResearchContext(BaseModel):
+      company: str
+
+  @pool.task("research")
+  async def research(agent_id: UUID, context: ResearchContext):
+      company = context.company  # Type-safe with IDE autocomplete
+  ```
+
+**Redis URL now required**
+- `redis_url` defaults to `None` and must be explicitly configured via `REDIS_URL`
+- Prevents accidental connections to wrong Redis instances
+
+### New Features
+
+**Configurable activity messages**
+- Activity status messages are configurable via environment variables:
+  ```bash
+  AGENTEXEC_ACTIVITY_MESSAGE_CREATE="Waiting to start."
+  AGENTEXEC_ACTIVITY_MESSAGE_STARTED="Task started."
+  AGENTEXEC_ACTIVITY_MESSAGE_COMPLETE="Task completed successfully."
+  AGENTEXEC_ACTIVITY_MESSAGE_ERROR="Task failed with error: {error}"
+  ```
+
+**Improved Task architecture**
+- `Task` is now the primary execution object with `execute()` method
+- `TaskDefinition` handles registration metadata and context class inference
+- Full lifecycle management (QUEUED → RUNNING → COMPLETE/ERROR) encapsulated in `Task.execute()`
+
+**Better SQLAlchemy session management**
+- New `scoped_session` pattern for worker processes
+- Proper session cleanup on worker shutdown
+
+### Internal Improvements
+
+- Switched to async Redis client (`redis.asyncio`)
+- Consolidated cleanup code in worker `_run()` method
+- Removed unused `debug` config option
+
+## v0.1.0
+
+Initial release.

{agentexec-0.1.2 → agentexec-0.1.4}/PKG-INFO

@@ -1,21 +1,21 @@
 Metadata-Version: 2.4
 Name: agentexec
-Version: 0.1.2
+Version: 0.1.4
 Summary: Production-ready orchestration for OpenAI Agents with Redis-backed coordination, activity tracking, and workflow management
 Project-URL: Homepage, https://github.com/Agent-CI/agentexec
 Project-URL: Documentation, https://github.com/Agent-CI/agentexec#readme
 Project-URL: Repository, https://github.com/Agent-CI/agentexec
 Project-URL: Issues, https://github.com/Agent-CI/agentexec/issues
-Author-email: Agent CI <hello@agentci.com>, Travis Dent <root@a10k.co>
+Author-email: Agent CI <hello@agent-ci.com>, Travis Dent <root@a10k.co>
 License: MIT
 Keywords: agents,async,background-workers,openai,orchestration,redis
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
-Requires-Python: >=3.11
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.12
 Requires-Dist: openai-agents>=0.1.0
 Requires-Dist: pydantic-settings>=2.5.0
 Requires-Dist: pydantic>=2.12.0
@@ -25,6 +25,12 @@ Description-Content-Type: text/markdown
 
 # `agentexec`
 
+[![PyPI](https://img.shields.io/pypi/v/agentexec?color=blue)](https://pypi.org/project/agentexec/)
+[![Python](https://img.shields.io/badge/python-3.12+-blue)](https://www.python.org/)
+[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+[![Type Checked](https://img.shields.io/badge/type%20checked-ty-blue)](https://github.com/astral-sh/ty)
+[![Code Style](https://img.shields.io/badge/code%20style-ruff-orange)](https://github.com/astral-sh/ruff)
+
 **Production-ready orchestration for OpenAI Agents SDK** with Redis-backed task queues, SQLAlchemy activity tracking, and multiprocessing worker pools.
 
 Build reliable, scalable AI agent applications with automatic lifecycle management, progress tracking, and fault tolerance.
@@ -61,7 +67,7 @@ uv add agentexec
 ```
 
 **Requirements:**
-- Python 3.11+
+- Python 3.12+
 - Redis (for task queue)
 - SQLAlchemy-compatible database (for activity tracking)
 - Agents that you want to parallelize!
@@ -82,16 +88,16 @@ from sqlalchemy import create_engine
 import agentexec as ax
 
 
-# Define typed context for your task
+# agentexec uses Pydantic models for input and outuput schemas
 class ResearchContext(BaseModel):
     company: str
 
 
-# Database for activity tracking (share with your app)
+# Database for activity tracking (share with the rest of your app)
 engine = create_engine("sqlite:///agents.db")
 
 # Create worker pool
-pool = ax.WorkerPool(engine=engine)
+pool = ax.Pool(engine=engine)
 
 
 @pool.task("research_company")
@@ -99,45 +105,34 @@ async def research_company(agent_id: UUID, context: ResearchContext) -> None:
     """Background task that runs an AI agent."""
     runner = ax.OpenAIRunner(
         agent_id=agent_id,
-        max_turns_recovery=True,
     )
 
     agent = Agent(
         name="Research Agent",
         instructions=(
-            f"Research {context.company}.\n"  # Typed access!
-            "\n"
-            f"{runner.prompts.report_status}"
+            f"Research {context.company}.\n"
+            runner.prompts.report_status
         ),
         tools=[
-            runner.tools.report_status,
+            runner.tools.report_status,  # Automatically associated with agent_id
         ],
-        model="gpt-4o",
+        model="gpt-5",
     )
 
     result = await runner.run(
         agent,
         input="Start research",
-        max_turns=15,
     )
-    print(f"Done! {result.final_output}")
+    print(f"Done! {result.final_output}")  # Native result object
 
 
 if __name__ == "__main__":
-    pool.start()  # Start workers
+    pool.start()  # Start worker process
 ```
 
 ### 2. Queue Tasks from Your Application
 
 ```python
-import agentexec as ax
-from pydantic import BaseModel
-
-
-class ResearchContext(BaseModel):
-    company: str
-
-
 # Enqueue a task (from your async API handler, etc.)
 task = await ax.enqueue(
     "research_company",
@@ -186,12 +181,19 @@ Agents can report their own progress using a built-in tool:
 ```python
 agent = Agent(
     instructions=f"Do research. {runner.prompts.report_status}",
-    tools=[runner.tools.report_status],  # Agent can call this
+    tools=[runner.tools.report_status],  # Agent passes a short message and percentage
 )
 
 # Agent will report: "Gathering data" (40%), "Analyzing results" (80%), etc.
 ```
 
+### Explicit Reporting
+Manually update activity status and progress from within your task:
+
+```python
+ax.activity.update(agent_id, "Starting data collection", percentage=10)
+```
+
 ### Max Turns Recovery
 
 Automatically handle conversation limits with graceful wrap-up:
@@ -202,6 +204,10 @@ runner = ax.OpenAIRunner(
     max_turns_recovery=True,
     wrap_up_prompt="Please summarize your findings.",
 )
+agent = Agent(
+    instructions="Research the topic thoroughly."
+)
+result = await runner.run(agent, max_turns=5)
 
 # If agent hits max turns, runner automatically:
 # 1. Catches MaxTurnsExceeded
@@ -226,32 +232,76 @@ await ax.enqueue("batch_job", context, priority=ax.Priority.LOW)
 Orchestrate multi-step workflows with parallel task execution:
 
 ```python
-import agentexec as ax
-
-pipeline = ax.Pipeline(pool=pool)
+pipeline = ax.Pipeline(pool)
 
 
 class MyPipeline(pipeline.Base):
-    @pipeline.step(0)
-    async def parallel_research(self, input_ctx: InputContext):
+    @pipeline.step(0, "brand and market research")
+    async def parallel_research(self, context: InputContext):
         """Run multiple tasks in parallel."""
-        task1 = await ax.enqueue("research_brand", BrandContext(...))
-        task2 = await ax.enqueue("research_market", MarketContext(...))
-        return await ax.gather(task1, task2)
+        brand_task = await ax.enqueue("research_brand", context)
+        market_task = await ax.enqueue("research_market", context)
+        return await ax.gather(brand_task, market_task)
 
-    @pipeline.step(1)
-    async def analyze(self, brand_result, market_result):
+    @pipeline.step(1, "research analysis")
+    async def analyze(self, brand_result: BrandResult, market_result: MarketResult) -> AnalysisResult:
         """Combine results from previous step."""
-        task = await ax.enqueue("analyze", AnalysisContext(...))
-        return await ax.get_result(task.agent_id)
+        task = await ax.enqueue("analyze_research", AnalysisContext(
+            brand=brand_result,
+            market=market_result,
+        ))
+        return await ax.get_result(task)
+
 
+# Queue to worker (non-blocking, activity tracked automatically)
+task = await pipeline.enqueue(context=InputContext(company="Anthropic"))
 
-# Run the pipeline
-result = await pipeline.run(context=InputContext(company="Anthropic"))
+# Or run inline (blocking)
+result = await pipeline.run(None, InputContext(company="Anthropic"))
 ```
 
 See **[examples/openai-agents-fastapi/pipeline.py](examples/openai-agents-fastapi/pipeline.py)** for a complete example.
 
+### Dynamic Fan-Out with Tracker
+
+Coordinate tasks that are queued dynamically and trigger a follow-up when all complete:
+
+```python
+import uuid
+
+tracker = ax.Tracker("research", uuid.uuid4())
+
+@pool.task("research")
+async def research(agent_id: UUID, context: ResearchContext) -> ResearchResult:
+    runner = ax.OpenAIRunner(agent_id=agent_id)
+    agent = Agent(
+        name="Research Agent",
+        instructions="...",
+        tools=[save_research],
+        output_type=ResearchResult,
+    )
+    return await runner.run(agent, input=f"Research {context.company}")
+
+
+@pool.task("aggregate")
+async def aggregate(agent_id: UUID, context: AggregateContext) -> None:
+    # ... aggregate results ...
+
+@function_tool
+async def queue_research(company: str) -> None:
+    tracker.incr()
+    await ax.enqueue("research", ResearchContext(company=company, batch_id=batch_id))
+
+@function_tool
+async def save_research(context: ResearchResult) -> None:
+    # save_research_to_database(context)
+    tracker.decr()
+    if tracker.complete:
+        await ax.enqueue("aggregate", ...)
+
+
+```
+
 ---
 
 ## Full Example: FastAPI Integration
@@ -317,7 +367,7 @@ class MyContext(BaseModel):
     param: str
 
 
-pool = ax.WorkerPool(engine=engine)
+pool = ax.Pool(engine=engine)
 
 
 @pool.task("task_name")
@@ -389,11 +439,54 @@ AgentExec creates two tables (prefix configurable):
 - `activity_id` - Foreign key to activity
 - `message` - Log message
 - `status` - QUEUED, RUNNING, COMPLETE, ERROR, CANCELED
-- `completion_percentage` - Progress (0-100)
+- `percentage` - Progress (0-100)
 - `created_at` - When log was created
 
 ---
 
+## Docker Deployment
+
+Deploy workers using the official Docker image:
+
+### 1. Create your worker Dockerfile
+
+```dockerfile
+FROM ghcr.io/agent-ci/agentexec-worker:latest
+
+COPY ./src /app/src
+ENV AGENTEXEC_WORKER_MODULE=src.worker
+```
+
+### 2. Create your worker module
+
+```python
+# src/worker.py
+import os
+import agentexec as ax
+
+pool = ax.Pool(database_url=os.environ["DATABASE_URL"])
+
+@pool.task("my_task")
+async def my_task(agent_id, context):
+    # Your task implementation
+    pass
+```
+
+### 3. Run with Docker
+
+```bash
+docker build -t my-worker .
+docker run \
+  -e DATABASE_URL=postgresql://... \
+  -e REDIS_URL=redis://... \
+  -e OPENAI_API_KEY=sk-... \
+  my-worker
+```
+
+See **[docker/README.md](docker/README.md)** for full documentation including Docker Compose examples.
+
+---
+
 ## Development
 
 ```bash
@@ -408,7 +501,7 @@ uv sync
 uv run pytest
 
 # Type checking
-uv run mypy src/agentexec
+uv run ty check
 
 # Linting
 uv run ruff check src/