agentexec 0.1.5__tar.gz → 0.1.7__tar.gz

agentexec-0.1.7/.claude/settings.local.json

@@ -0,0 +1,7 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(uv run:*)"
+    ]
+  }
+}

{agentexec-0.1.5 → agentexec-0.1.7}/CHANGELOG.md

@@ -1,5 +1,58 @@
 # Changelog
 
+## v0.1.7
+
+### New Features
+
+**Scheduled tasks with cron expressions**
+- `@pool.schedule("task_name", "*/5 * * * *")` decorator registers and schedules a task in one step
+- `pool.add_schedule()` for imperative scheduling of already-registered tasks
+- Cron expressions evaluated in configurable timezone (`AGENTEXEC_SCHEDULER_TIMEZONE`, default UTC)
+- Repeat budget: `-1` for forever (default), `0` for one-shot, `N` for N more executions
+- Scheduler runs automatically inside `pool.run()` — no extra setup needed
+- Idempotent registration: keyed by task name, so restarts and multiple pool instances overwrite instead of duplicating
+- Clock-drift resilient: next run computed from intended anchor time, not wall clock
+- Skips missed intervals after downtime instead of enqueuing a burst of catch-up tasks
+- New `croniter` dependency for cron expression parsing
+
+### Improvements
+
+**State backend sorted set operations**
+- Added `zadd()`, `zrangebyscore()`, `zrem()` to `StateBackend` protocol and Redis implementation
+- Used internally by the scheduler for efficient due-task polling
+
+## v0.1.6
+
+### New Features
+
+**Task-level distributed locking**
+- New `lock_key` parameter on `@pool.task()` and `pool.add_task()` for sequential execution of tasks sharing state
+- String template evaluated against context fields (e.g., `lock_key="user:{user_id}"`)
+- Workers acquire a Redis lock before execution; tasks requeue automatically on contention
+- Lock released in `finally` block on completion or error
+- Configurable TTL via `AGENTEXEC_LOCK_TTL` (default 1800s) as safety net for worker process death
+- Note: strict FIFO ordering is not guaranteed between tasks sharing the same lock key
+
+**Activity metadata for multi-tenancy**
+- Attach arbitrary metadata when creating activities (e.g., `metadata={"organization_id": "org-123"}`)
+- Filter activities by metadata in `activity.list()` and `activity.detail()`
+- Metadata accessible as attribute for programmatic use but excluded from API serialization by default to prevent accidental tenant info leakage
+
+### Improvements
+
+**Redis cleanup on shutdown**
+- `state.clear_keys()` removes all agentexec-prefixed keys and the task queue on shutdown
+- Prevents stale tasks from being picked up on restart
+
+**State backend lock primitives**
+- Added `acquire_lock()` and `release_lock()` to `StateBackend` protocol
+- Redis implementation uses atomic `SET NX EX` / `DELETE`
+
+### Testing
+
+- Added `test_task_locking.py` with 16 tests covering lock acquisition, release, requeue, and template evaluation
+- Fixed `ty` type checker errors in `test_activity_tracking.py` (added narrowing guards for `Activity | None`)
+
 ## v0.1.5
 
 ### New Features

{agentexec-0.1.5 → agentexec-0.1.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentexec
-Version: 0.1.5
+Version: 0.1.7
 Summary: Production-ready orchestration for OpenAI Agents with Redis-backed coordination, activity tracking, and workflow management
 Project-URL: Homepage, https://github.com/Agent-CI/agentexec
 Project-URL: Documentation, https://github.com/Agent-CI/agentexec#readme
@@ -16,6 +16,7 @@ Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Requires-Python: >=3.12
+Requires-Dist: croniter>=6.0.0
 Requires-Dist: openai-agents>=0.1.0
 Requires-Dist: pydantic-settings>=2.5.0
 Requires-Dist: pydantic>=2.12.0
@@ -163,6 +164,25 @@ That's it. Tasks are queued to Redis, workers process them in parallel, progress
 
 ## Supported Patterns
 
+### Activity Metadata (Multi-Tenancy)
+
+Attach arbitrary metadata when enqueueing tasks for filtering and tenant isolation:
+
+```python
+task = await ax.enqueue(
+    "process_document",
+    context,
+    metadata={"organization_id": "org-123", "user_id": "user-456"}
+)
+
+# Filter activities by metadata
+activities = ax.activity.list(db, metadata_filter={"organization_id": "org-123"})
+detail = ax.activity.detail(db, agent_id, metadata_filter={"organization_id": "org-123"})
+
+# Access metadata programmatically (excluded from API serialization by default)
+org_id = detail.metadata["organization_id"]
+```
+
 ### Automatic Activity Tracking
 
 Every task gets full lifecycle tracking without manual updates:
@@ -195,6 +215,50 @@ Update progress explicitly from your task:
 ax.activity.update(agent_id, "Processing batch 3 of 10", percentage=30)
 ```
 
+### Task Locking
+
+When multiple tasks of the same type are queued for the same user, they may need to run sequentially because each task reads and writes shared state. Use `lock_key` to ensure only one task with the same evaluated key runs at a time:
+
+```python
+@pool.task("associate_observation", lock_key="user:{user_id}")
+async def associate(agent_id: UUID, context: ObservationContext):
+    ...
+
+# Or with add_task()
+pool.add_task("associate_observation", handler, lock_key="user:{user_id}")
+```
+
+The `lock_key` is a string template evaluated against the task context fields. When a worker dequeues a task whose lock is held, it puts the task back at the end of the queue and moves on. The lock is released automatically when the task completes or errors.
+
+The lock TTL (`AGENTEXEC_LOCK_TTL`, default 1800s) is a safety net for worker process death — locks are always explicitly released on task completion or error. Set this higher than your longest expected task duration.
+
+**Note:** When a task is requeued due to a held lock, it goes to the back of the queue. This means strict FIFO ordering is not guaranteed between tasks sharing the same lock key — if tasks T2 and T3 are both waiting on T1's lock, either could run next after T1 completes.
+
+### Scheduled Tasks
+
+Run tasks on a recurring interval using cron expressions:
+
+```python
+# Decorator — registers the task and schedules it in one step
+@pool.schedule("refresh_cache", "*/5 * * * *")
+async def refresh(agent_id: UUID, context: RefreshContext):
+    ...
+
+# With context and repeat limit
+@pool.schedule("sync_users", "0 * * * *", context=SyncContext(full=True), repeat=3)
+async def sync(agent_id: UUID, context: SyncContext):
+    ...
+```
+
+For tasks registered separately, use `pool.add_schedule()`:
+
+```python
+pool.add_schedule("refresh_cache", "*/5 * * * *", RefreshContext(scope="all"))
+pool.add_schedule("refresh_cache", "0 * * * *", RefreshContext(scope="users"), repeat=3)
+```
+
+The scheduler runs automatically inside `pool.run()`. Cron expressions are evaluated in the configured timezone (`AGENTEXEC_SCHEDULER_TIMEZONE`, default UTC) so schedules read naturally regardless of server timezone. Next-run times are computed from the intended anchor time, not wall clock, to prevent cumulative drift.
+
 ### Priority Queue
 
 Control task execution order:
@@ -411,6 +475,7 @@ Activity tracking uses SQLAlchemy with two tables:
 **`agentexec_activity`** - Main activity records
 - `agent_id` - Unique identifier (UUID)
 - `agent_type` - Task name
+- `metadata` - JSON field for custom data (e.g., tenant info)
 - `created_at`, `updated_at` - Timestamps
 
 **`agentexec_activity_log`** - Status and progress
@@ -600,7 +665,15 @@ pool = ax.Pool(database_url="postgresql://...")
 @pool.task("name")
 async def handler(agent_id: UUID, context: MyContext) -> None: ...
 
-pool.run()       # Blocking - runs workers
+@pool.task("name", lock_key="user:{user_id}")  # Sequential per user
+async def locked(agent_id: UUID, context: MyContext) -> None: ...
+
+@pool.schedule("name", "*/5 * * * *")  # Register + schedule in one step
+async def scheduled(agent_id: UUID, context: MyContext) -> None: ...
+
+pool.add_schedule("name", "0 * * * *", MyContext(), repeat=3)  # Schedule separately
+
+pool.run()       # Blocking - runs workers + scheduler
 pool.start()     # Non-blocking - starts workers in background
 pool.shutdown()  # Graceful shutdown
 ```
@@ -700,6 +773,12 @@ AGENTEXEC_TABLE_PREFIX=agentexec_
 # Results
 AGENTEXEC_RESULT_TTL=3600
 
+# Task locking
+AGENTEXEC_LOCK_TTL=1800
+
+# Scheduling
+AGENTEXEC_SCHEDULER_TIMEZONE=UTC
+
 # State backend
 AGENTEXEC_STATE_BACKEND=agentexec.state.redis_backend
 AGENTEXEC_KEY_PREFIX=agentexec
@@ -758,4 +837,5 @@ MIT License - see [LICENSE](LICENSE) for details.
 - **npm**: [agentexec-ui](https://www.npmjs.com/package/agentexec-ui)
 - **Documentation**: [docs/](docs/)
 - **Example App**: [examples/openai-agents-fastapi/](examples/openai-agents-fastapi/)
+- **Multi-Tenancy Example**: [examples/multi-tenancy/](examples/multi-tenancy/)
 - **Issues**: [GitHub Issues](https://github.com/Agent-CI/agentexec/issues)

{agentexec-0.1.5 → agentexec-0.1.7}/README.md

@@ -138,6 +138,25 @@ That's it. Tasks are queued to Redis, workers process them in parallel, progress
 
 ## Supported Patterns
 
+### Activity Metadata (Multi-Tenancy)
+
+Attach arbitrary metadata when enqueueing tasks for filtering and tenant isolation:
+
+```python
+task = await ax.enqueue(
+    "process_document",
+    context,
+    metadata={"organization_id": "org-123", "user_id": "user-456"}
+)
+
+# Filter activities by metadata
+activities = ax.activity.list(db, metadata_filter={"organization_id": "org-123"})
+detail = ax.activity.detail(db, agent_id, metadata_filter={"organization_id": "org-123"})
+
+# Access metadata programmatically (excluded from API serialization by default)
+org_id = detail.metadata["organization_id"]
+```
+
 ### Automatic Activity Tracking
 
 Every task gets full lifecycle tracking without manual updates:
@@ -170,6 +189,50 @@ Update progress explicitly from your task:
 ax.activity.update(agent_id, "Processing batch 3 of 10", percentage=30)
 ```
 
+### Task Locking
+
+When multiple tasks of the same type are queued for the same user, they may need to run sequentially because each task reads and writes shared state. Use `lock_key` to ensure only one task with the same evaluated key runs at a time:
+
+```python
+@pool.task("associate_observation", lock_key="user:{user_id}")
+async def associate(agent_id: UUID, context: ObservationContext):
+    ...
+
+# Or with add_task()
+pool.add_task("associate_observation", handler, lock_key="user:{user_id}")
+```
+
+The `lock_key` is a string template evaluated against the task context fields. When a worker dequeues a task whose lock is held, it puts the task back at the end of the queue and moves on. The lock is released automatically when the task completes or errors.
+
+The lock TTL (`AGENTEXEC_LOCK_TTL`, default 1800s) is a safety net for worker process death — locks are always explicitly released on task completion or error. Set this higher than your longest expected task duration.
+
+**Note:** When a task is requeued due to a held lock, it goes to the back of the queue. This means strict FIFO ordering is not guaranteed between tasks sharing the same lock key — if tasks T2 and T3 are both waiting on T1's lock, either could run next after T1 completes.
+
+### Scheduled Tasks
+
+Run tasks on a recurring interval using cron expressions:
+
+```python
+# Decorator — registers the task and schedules it in one step
+@pool.schedule("refresh_cache", "*/5 * * * *")
+async def refresh(agent_id: UUID, context: RefreshContext):
+    ...
+
+# With context and repeat limit
+@pool.schedule("sync_users", "0 * * * *", context=SyncContext(full=True), repeat=3)
+async def sync(agent_id: UUID, context: SyncContext):
+    ...
+```
+
+For tasks registered separately, use `pool.add_schedule()`:
+
+```python
+pool.add_schedule("refresh_cache", "*/5 * * * *", RefreshContext(scope="all"))
+pool.add_schedule("refresh_cache", "0 * * * *", RefreshContext(scope="users"), repeat=3)
+```
+
+The scheduler runs automatically inside `pool.run()`. Cron expressions are evaluated in the configured timezone (`AGENTEXEC_SCHEDULER_TIMEZONE`, default UTC) so schedules read naturally regardless of server timezone. Next-run times are computed from the intended anchor time, not wall clock, to prevent cumulative drift.
+
 ### Priority Queue
 
 Control task execution order:
@@ -386,6 +449,7 @@ Activity tracking uses SQLAlchemy with two tables:
 **`agentexec_activity`** - Main activity records
 - `agent_id` - Unique identifier (UUID)
 - `agent_type` - Task name
+- `metadata` - JSON field for custom data (e.g., tenant info)
 - `created_at`, `updated_at` - Timestamps
 
 **`agentexec_activity_log`** - Status and progress
@@ -575,7 +639,15 @@ pool = ax.Pool(database_url="postgresql://...")
 @pool.task("name")
 async def handler(agent_id: UUID, context: MyContext) -> None: ...
 
-pool.run()       # Blocking - runs workers
+@pool.task("name", lock_key="user:{user_id}")  # Sequential per user
+async def locked(agent_id: UUID, context: MyContext) -> None: ...
+
+@pool.schedule("name", "*/5 * * * *")  # Register + schedule in one step
+async def scheduled(agent_id: UUID, context: MyContext) -> None: ...
+
+pool.add_schedule("name", "0 * * * *", MyContext(), repeat=3)  # Schedule separately
+
+pool.run()       # Blocking - runs workers + scheduler
 pool.start()     # Non-blocking - starts workers in background
 pool.shutdown()  # Graceful shutdown
 ```
@@ -675,6 +747,12 @@ AGENTEXEC_TABLE_PREFIX=agentexec_
 # Results
 AGENTEXEC_RESULT_TTL=3600
 
+# Task locking
+AGENTEXEC_LOCK_TTL=1800
+
+# Scheduling
+AGENTEXEC_SCHEDULER_TIMEZONE=UTC
+
 # State backend
 AGENTEXEC_STATE_BACKEND=agentexec.state.redis_backend
 AGENTEXEC_KEY_PREFIX=agentexec
@@ -733,4 +811,5 @@ MIT License - see [LICENSE](LICENSE) for details.
 - **npm**: [agentexec-ui](https://www.npmjs.com/package/agentexec-ui)
 - **Documentation**: [docs/](docs/)
 - **Example App**: [examples/openai-agents-fastapi/](examples/openai-agents-fastapi/)
+- **Multi-Tenancy Example**: [examples/multi-tenancy/](examples/multi-tenancy/)
 - **Issues**: [GitHub Issues](https://github.com/Agent-CI/agentexec/issues)

agentexec-0.1.7/examples/multi-tenancy/README.md

@@ -0,0 +1,92 @@
+# Multi-Tenancy with Activity Metadata
+
+This example demonstrates how to use activity metadata for multi-tenant applications.
+
+## Overview
+
+When building multi-tenant applications, you often need to:
+1. Associate background tasks with specific organizations/tenants
+2. Filter activity views to only show tasks belonging to the current tenant
+3. Ensure proper data isolation between tenants
+
+The `metadata` parameter on `ax.enqueue()` and `pipeline.enqueue()` enables this by attaching arbitrary key-value pairs to activity records.
+
+## Usage
+
+### Enqueueing with Metadata
+
+```python
+import agentexec as ax
+
+# Enqueue a task with organization context
+task = await ax.enqueue(
+    "process_document",
+    DocumentContext(file_id="doc-123"),
+    metadata={"organization_id": "org-456", "user_id": "user-789"}
+)
+```
+
+### Filtering Activities by Metadata
+
+```python
+from agentexec import activity
+
+# List only activities for a specific organization
+activities = activity.list(
+    session,
+    metadata_filter={"organization_id": "org-456"}
+)
+
+# Get activity detail with tenant validation
+# Returns None if the activity doesn't belong to this organization
+detail = activity.detail(
+    session,
+    agent_id="...",
+    metadata_filter={"organization_id": "org-456"}
+)
+```
+
+### Pipeline Example
+
+```python
+pipeline = ax.Pipeline(pool)
+
+class DocumentPipeline(pipeline.Base):
+    @pipeline.step(0)
+    async def extract(self, ctx: DocumentContext) -> ExtractedData:
+        ...
+
+# Enqueue pipeline with metadata
+task = await pipeline.enqueue(
+    context=DocumentContext(file_id="doc-123"),
+    metadata={"organization_id": "org-456"}
+)
+```
+
+## Running the Example
+
+```bash
+# Install dependencies
+pip install agentexec
+
+# Run the example
+python example.py
+```
+
+## Database Schema
+
+The metadata is stored as a JSON column on the `agentexec_activity` table:
+
+```sql
+ALTER TABLE agentexec_activity ADD COLUMN metadata JSON;
+```
+
+For PostgreSQL, this maps to JSONB which supports efficient filtering.
+
+## Notes
+
+- Metadata is immutable once set at enqueue time
+- Filtering uses exact string matching on metadata values
+- **Metadata is excluded from API serialization by default** to prevent accidental leakage of tenant info
+- Access metadata programmatically via the `.metadata` attribute (e.g., `activity.metadata`)
+- To include metadata in API responses, explicitly add it to your response model

agentexec-0.1.7/examples/multi-tenancy/example.py

@@ -0,0 +1,188 @@
+"""Multi-tenancy example demonstrating activity metadata filtering.
+
+This example shows how to:
+1. Attach metadata (like organization_id) when enqueueing tasks
+2. Filter activities by metadata for tenant isolation
+3. Use metadata in both list and detail views
+"""
+
+import asyncio
+from uuid import UUID
+
+from pydantic import BaseModel
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+
+import agentexec as ax
+
+
+# --- Models ---
+
+
+class DocumentContext(BaseModel):
+    """Input context for document processing."""
+
+    file_id: str
+    filename: str
+
+
+class ProcessedDocument(BaseModel):
+    """Result of document processing."""
+
+    file_id: str
+    word_count: int
+    summary: str
+
+
+# --- Setup ---
+
+# Create in-memory SQLite database for demo
+engine = create_engine("sqlite:///multi_tenant_demo.db", echo=False)
+SessionLocal = sessionmaker(bind=engine)
+
+# Create tables
+ax.Base.metadata.create_all(bind=engine)
+
+# Create worker pool
+pool = ax.Pool(engine=engine)
+
+
+# --- Task Definition ---
+
+
+@pool.task("process_document")
+async def process_document(
+    *,
+    agent_id: UUID,
+    context: DocumentContext,
+) -> ProcessedDocument:
+    """Simulate document processing."""
+    # Simulate some work
+    ax.activity.update(agent_id, "Extracting text...", percentage=25)
+    await asyncio.sleep(0.1)
+
+    ax.activity.update(agent_id, "Analyzing content...", percentage=50)
+    await asyncio.sleep(0.1)
+
+    ax.activity.update(agent_id, "Generating summary...", percentage=75)
+    await asyncio.sleep(0.1)
+
+    return ProcessedDocument(
+        file_id=context.file_id,
+        word_count=1234,
+        summary=f"Summary of {context.filename}",
+    )
+
+
+# --- Demo Functions ---
+
+
+async def enqueue_tasks_for_tenants():
+    """Enqueue tasks for different organizations."""
+    print("\n=== Enqueueing Tasks ===\n")
+
+    # Organization A: Enqueue 2 tasks
+    task1 = await ax.enqueue(
+        "process_document",
+        DocumentContext(file_id="doc-001", filename="report.pdf"),
+        metadata={"organization_id": "org-A", "user_id": "user-1"},
+    )
+    print(f"Org A - Task 1: {task1.agent_id}")
+
+    task2 = await ax.enqueue(
+        "process_document",
+        DocumentContext(file_id="doc-002", filename="invoice.pdf"),
+        metadata={"organization_id": "org-A", "user_id": "user-2"},
+    )
+    print(f"Org A - Task 2: {task2.agent_id}")
+
+    # Organization B: Enqueue 1 task
+    task3 = await ax.enqueue(
+        "process_document",
+        DocumentContext(file_id="doc-003", filename="contract.pdf"),
+        metadata={"organization_id": "org-B", "user_id": "user-3"},
+    )
+    print(f"Org B - Task 1: {task3.agent_id}")
+
+    return task1, task2, task3
+
+
+def list_activities_for_tenant(org_id: str):
+    """List activities filtered by organization."""
+    print(f"\n=== Activities for {org_id} ===\n")
+
+    with SessionLocal() as session:
+        result = ax.activity.list(
+            session,
+            metadata_filter={"organization_id": org_id},
+        )
+
+        print(f"Total activities: {result.total}")
+        for item in result.items:
+            print(f"  - {item.agent_id}: {item.agent_type} ({item.status})")
+            print(f"    Metadata: {item.metadata}")
+
+
+def get_activity_detail_with_tenant_check(agent_id: UUID, org_id: str):
+    """Get activity detail with tenant validation."""
+    print(f"\n=== Detail for {agent_id} (checking {org_id}) ===\n")
+
+    with SessionLocal() as session:
+        # This returns None if the activity doesn't belong to the org
+        detail = ax.activity.detail(
+            session,
+            agent_id,
+            metadata_filter={"organization_id": org_id},
+        )
+
+        if detail:
+            print(f"Found: {detail.agent_type}")
+            print(f"Metadata: {detail.metadata}")
+            print(f"Logs: {len(detail.logs)} entries")
+        else:
+            print(f"Not found (or doesn't belong to {org_id})")
+
+
+async def main():
+    """Run the multi-tenancy demo."""
+    print("Multi-Tenancy Demo with Activity Metadata")
+    print("=" * 50)
+
+    # 1. Enqueue tasks for different organizations
+    task1, task2, task3 = await enqueue_tasks_for_tenants()
+
+    # 2. List all activities (no filter)
+    print("\n=== All Activities (no filter) ===\n")
+    with SessionLocal() as session:
+        all_activities = ax.activity.list(session)
+        print(f"Total: {all_activities.total}")
+        for item in all_activities.items:
+            print(f"  - {item.agent_type}: {item.metadata}")
+
+    # 3. List activities filtered by organization
+    list_activities_for_tenant("org-A")  # Should show 2 tasks
+    list_activities_for_tenant("org-B")  # Should show 1 task
+    list_activities_for_tenant("org-C")  # Should show 0 tasks
+
+    # 4. Detail view with tenant validation
+    # Try to access Org A's task as Org A (should work)
+    get_activity_detail_with_tenant_check(task1.agent_id, "org-A")
+
+    # Try to access Org A's task as Org B (should return None)
+    get_activity_detail_with_tenant_check(task1.agent_id, "org-B")
+
+    # 5. Filter by multiple metadata fields
+    print("\n=== Filter by org AND user ===\n")
+    with SessionLocal() as session:
+        result = ax.activity.list(
+            session,
+            metadata_filter={"organization_id": "org-A", "user_id": "user-1"},
+        )
+        print(f"Found {result.total} activities for org-A + user-1")
+
+    print("\n" + "=" * 50)
+    print("Demo complete!")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

{agentexec-0.1.5 → agentexec-0.1.7}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "agentexec"
-version = "0.1.5"
+version = "0.1.7"
 description = "Production-ready orchestration for OpenAI Agents with Redis-backed coordination, activity tracking, and workflow management"
 readme = "README.md"
 requires-python = ">=3.12"
@@ -25,6 +25,7 @@ dependencies = [
     "pydantic-settings>=2.5.0",
     "sqlalchemy>=2.0.44",
     "openai-agents>=0.1.0",
+    "croniter>=6.0.0",
 ]