devsper 2.3.0__tar.gz → 2.4.0__tar.gz

{devsper-2.3.0 → devsper-2.4.0}/CHANGELOG.md

@@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [2.4.0] — 2026-03-31
+
+### Added
+
+- **Distributed runtime extension layer** — Added `devsper.distributed` package with controller and worker-runtime wrappers for clean orchestration composition.
+- **Agent pool** — Added `runtime/agent_pool.py` with reusable, worker-aware concurrent agent leasing (`acquire_agent`, `release_agent`, `run_agent`, `run_parallel`).
+- **Model router (runtime layer)** — Added `runtime/model_router.py` for planning/reasoning/validation model routing with fallback chains.
+- **Speculative planner (runtime layer)** — Added `runtime/speculative_planner.py` to predict and stage speculative successor tasks.
+- **Runtime HITL manager** — Added `runtime/hitl.py` for pause/resume and explicit human-input request lifecycle handling.
+- **Execution graph upgrades** — Extended runtime execution graph with worker assignment metadata and distributed lineage tracking.
+
+### Changed
+
+- **Executor composition** — Runtime executor now integrates AgentPool, ModelRouter, speculative planning hooks, and distributed assignment tracking while preserving default behavior.
+- **Task runner routing** — Task execution now supports pool-backed agent execution, scoped model routing, and fallback chain handling.
+- **Tool execution control** — Runtime ToolRunner now supports bounded concurrency, dependency-aware batching, per-call timeout, cancellation, and structured results.
+- **Event backpressure handling** — Runtime event stream now uses bounded queues with controlled overflow behavior.
+- **Runtime docs** — Updated runtime architecture docs in `devsper/README.md` with distributed flow, worker/controller lifecycle, and HITL/speculative execution details.
+
 ## [2.3.0] — 2026-03-30
 
 ### Added

{devsper-2.3.0 → devsper-2.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: devsper
-Version: 2.3.0
+Version: 2.4.0
 Summary: Orchestrate distributed swarms of AI agents that collaboratively solve complex tasks.
 Project-URL: Homepage, https://github.com/devsper-com/runtime
 Project-URL: Documentation, https://docs.devsper.com

devsper-2.4.0/devsper/README.md

@@ -0,0 +1,148 @@
+# Devsper Runtime Core
+
+Devsper runtime is built as a modular orchestration core with bounded concurrency, dynamic DAG mutation, and event-stream-driven observability.
+
+## Runtime Architecture Diagram
+
+```text
+User Task
+   |
+   v
+Swarm.run()
+   |
+   v
+Planner -> Scheduler (DAG)
+   |
+   v
+Runtime Executor ------------------------------+
+   |                                           |
+   +-> RuntimeStateManager                     |
+   +-> ExecutionGraph                          |
+   +-> RuntimeEventStream -> EventLog -> SSE  |
+   +-> TaskRunner                              |
+         +-> AgentRunner (optional stream-tool loop)
+               +-> Agent
+               +-> ToolRunner / tools.tool_runner
+```
+
+## Core Components
+
+- `swarm/swarm.py`
+  - Public entrypoint (`Swarm`), config/wiring, planning and execution bootstrap.
+- `runtime/executor.py`
+  - Event-driven scheduler loop, bounded parallel execution, cancellation propagation, dynamic task injection.
+- `runtime/state_manager.py`
+  - Concurrency-safe task state transitions and runtime DAG mutation entrypoint.
+- `runtime/execution_graph.py`
+  - Execution graph with lineage, edges, attempts, and status transitions.
+- `runtime/planner.py`
+  - Runtime wrapper for dynamic planner expansion and parent-child lineage.
+- `runtime/task_runner.py`
+  - Task lifecycle orchestration with scoped retries and fallback model handling.
+- `runtime/agent_runner.py`
+  - Async agent wrapper with optional streaming tool invocation loop.
+- `runtime/tool_runner.py`
+  - Parallel tool scheduler with bounded concurrency, dependency-aware batching, timeout, and cancellation.
+- `runtime/event_stream.py`
+  - In-process stream with queue backpressure policy (drop-oldest on overflow).
+- `runtime/retry.py`
+  - Retry scopes (`tool`, `agent`, `task`, `model_fallback`) and backoff policies.
+
+## Execution Lifecycle
+
+1. `Swarm.run()` creates root task, plans subtasks, and builds scheduler DAG.
+2. Executor emits `EXECUTOR_STARTED` and begins bounded task dispatch.
+3. Ready tasks transition to running through `RuntimeStateManager`.
+4. `TaskRunner` executes with scoped retries.
+5. `AgentRunner` may invoke tools (normal or streaming tool loop).
+6. Task completion/failure updates scheduler + execution graph.
+7. Adaptive mode can inject follow-up tasks at runtime.
+8. Executor emits `EXECUTOR_FINISHED` and `RUN_COMPLETED`.
+
+## Tool Calling Flow
+
+1. Agent determines tool intent from model output.
+2. Tool calls are parsed and scheduled.
+3. `ToolRunner.run_many(...)` executes calls in parallel with:
+   - `max_concurrency` semaphore
+   - dependency constraints (`depends_on`)
+   - per-call timeouts
+   - cancellation checks
+4. Results are isolated per call and fed back into agent loop.
+
+## Planner Flow
+
+1. Initial decomposition by `swarm/planner.py`.
+2. Runtime execution completes a task.
+3. `RuntimePlanner.expand(...)` optionally creates follow-up tasks.
+4. `RuntimeStateManager.add_tasks(...)` mutates DAG safely.
+5. `ExecutionGraph` records lineage from parent task.
+
+## Concurrency Model
+
+- Bounded executor workers (`worker_count`).
+- Bounded tool concurrency (`ToolRunner` semaphore).
+- Queue backpressure in event streaming (`max_queue_size` with controlled dropping).
+- Cooperative pause/resume and cancellation propagation across runtime loops.
+- Lock-guarded scheduler mutations via `RuntimeStateManager`.
+
+## Distributed Runtime Architecture
+
+```text
+Controller
+   |
+   v
+Worker Pool
+   |
+   v
+Worker Runtime
+   |
+   v
+Runtime Executor
+   |
+   v
+Agent Pool
+   |
+   v
+Tool Runner
+```
+
+- `distributed/controller.py`
+  - Worker registry, health state, load-aware assignment, reassignment hooks.
+- `distributed/worker_runtime.py`
+  - Worker-local runtime composition: `Executor`, `AgentPool`, `ModelRouter`, `ToolRunner`.
+- Existing multi-node transport/control remains in `nodes/controller.py` and `nodes/worker.py`.
+
+## Worker Lifecycle
+
+1. Worker registers with controller.
+2. Controller assigns tasks based on health and load.
+3. Worker executes task locally through runtime executor stack.
+4. Worker returns completion/failure.
+5. Controller updates worker/task state and reassigns on failure.
+
+## Controller Lifecycle
+
+1. Track worker registration and health.
+2. Assign ready tasks using load-aware strategy.
+3. Handle failures and perform retry/reassignment.
+4. Maintain global execution progress via event/log channels.
+
+## Agent Pool
+
+- `AgentPool` manages reusable agent instances per worker.
+- Supports `acquire_agent`, `release_agent`, `run_agent`, and `run_parallel`.
+- Enables worker-local, concurrent, reuse-based execution.
+
+## Speculative Execution
+
+- `SpeculativePlanner` predicts likely successor tasks.
+- Executor marks/schedules speculative tasks early.
+- Unused speculative branches can be cancelled on dependency failure.
+
+## HITL Flow
+
+1. Agent indicates human input is required.
+2. Runtime emits HITL/clarification event.
+3. Task pauses until response or timeout.
+4. Execution resumes or fails based on response policy.

{devsper-2.3.0 → devsper-2.4.0}/devsper/agents/agent.py

@@ -454,11 +454,24 @@ def _format_tools_section(tools: list | None = None) -> str:
         from devsper.tools.registry import list_tools
 
         tools = list_tools()
+    max_props = 8
     lines = []
     for t in tools:
         schema = getattr(t, "input_schema", None) or getattr(t, "schema", {}) or {}
+        schema_type = schema.get("type", "object")
+        required = schema.get("required", []) or []
+        props = schema.get("properties", {}) if isinstance(schema.get("properties", {}), dict) else {}
+        compact_props: dict[str, dict] = {}
+        for key in list(props.keys())[:max_props]:
+            p = props.get(key, {}) or {}
+            compact_props[key] = {"type": p.get("type", "string")}
+        compact_schema = {
+            "type": schema_type,
+            "required": required[:max_props],
+            "properties": compact_props,
+        }
         lines.append(f"- {t.name}: {t.description}")
-        lines.append(f"  input_schema: {json.dumps(schema)}")
+        lines.append(f"  input_schema: {json.dumps(compact_schema, separators=(',', ':'))}")
     return "\n".join(lines)
 
 

devsper-2.4.0/devsper/distributed/__init__.py

@@ -0,0 +1,7 @@
+"""Distributed runtime wrappers for controller/worker orchestration."""
+
+from devsper.distributed.controller import DistributedController
+from devsper.distributed.worker_runtime import WorkerRuntime
+
+__all__ = ["DistributedController", "WorkerRuntime"]
+

devsper-2.4.0/devsper/distributed/controller.py

@@ -0,0 +1,70 @@
+from __future__ import annotations
+
+import asyncio
+from dataclasses import dataclass
+
+from devsper.cluster.router import TaskRouter
+from devsper.types.task import Task
+
+
+@dataclass
+class WorkerState:
+    worker_id: str
+    healthy: bool = True
+    active_tasks: int = 0
+    max_workers: int = 1
+
+
+class DistributedController:
+    """Controller-side worker orchestration and task assignment."""
+
+    def __init__(self, router: TaskRouter | None = None) -> None:
+        self._router = router or TaskRouter()
+        self._workers: dict[str, WorkerState] = {}
+        self._task_assignments: dict[str, str] = {}
+        self._lock = asyncio.Lock()
+
+    async def register_worker(self, worker_id: str, max_workers: int = 1) -> None:
+        async with self._lock:
+            self._workers[worker_id] = WorkerState(
+                worker_id=worker_id,
+                healthy=True,
+                active_tasks=0,
+                max_workers=max(1, int(max_workers)),
+            )
+
+    async def mark_worker_unhealthy(self, worker_id: str) -> None:
+        async with self._lock:
+            if worker_id in self._workers:
+                self._workers[worker_id].healthy = False
+
+    async def health_check(self) -> dict[str, bool]:
+        async with self._lock:
+            return {wid: ws.healthy for wid, ws in self._workers.items()}
+
+    async def assign_task(self, task: Task) -> str | None:
+        async with self._lock:
+            candidates = [
+                ws
+                for ws in self._workers.values()
+                if ws.healthy and ws.active_tasks < ws.max_workers
+            ]
+            if not candidates:
+                return None
+            # Lightweight balancing by active load.
+            candidates.sort(key=lambda ws: (ws.active_tasks / max(1, ws.max_workers), ws.worker_id))
+            chosen = candidates[0]
+            chosen.active_tasks += 1
+            self._task_assignments[task.id] = chosen.worker_id
+            return chosen.worker_id
+
+    async def complete_task(self, task_id: str) -> None:
+        async with self._lock:
+            wid = self._task_assignments.pop(task_id, None)
+            if wid and wid in self._workers:
+                self._workers[wid].active_tasks = max(0, self._workers[wid].active_tasks - 1)
+
+    async def reassign_on_failure(self, task: Task, failed_worker_id: str) -> str | None:
+        await self.mark_worker_unhealthy(failed_worker_id)
+        return await self.assign_task(task)
+

devsper-2.4.0/devsper/distributed/worker_runtime.py

@@ -0,0 +1,43 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from devsper.agents.agent import Agent
+from devsper.runtime.agent_pool import AgentPool
+from devsper.runtime.executor import Executor
+from devsper.runtime.model_router import ModelRouter
+from devsper.runtime.tool_runner import ToolRunner
+from devsper.swarm.scheduler import Scheduler
+from devsper.utils.event_logger import EventLog
+
+
+@dataclass
+class WorkerRuntime:
+    """Worker-local runtime for agent/tool execution."""
+
+    scheduler: Scheduler
+    agent: Agent
+    event_log: EventLog
+    worker_id: str = "worker-local"
+    max_agents: int = 4
+    tool_concurrency: int = 4
+
+    def __post_init__(self) -> None:
+        self.agent_pool = AgentPool(lambda: self.agent, max_agents=self.max_agents)
+        self.model_router = ModelRouter(
+            planning_model=getattr(self.agent, "model_name", "mock"),
+            reasoning_model=getattr(self.agent, "model_name", "mock"),
+            validation_model=getattr(self.agent, "model_name", "mock"),
+        )
+        self.tool_runner = ToolRunner(parallelism=self.tool_concurrency)
+        self.executor = Executor(
+            scheduler=self.scheduler,
+            agent=self.agent,
+            event_log=self.event_log,
+            worker_count=self.max_agents,
+        )
+
+    async def run_task_queue(self) -> dict[str, str]:
+        await self.executor.run()
+        return self.scheduler.get_results()
+

devsper-2.4.0/devsper/runtime/__init__.py

@@ -0,0 +1,34 @@
+"""Runtime utilities: replay, telemetry, visualization."""
+
+from devsper.runtime.replay import replay_execution
+from devsper.runtime.telemetry import collect_telemetry, print_telemetry_summary
+from devsper.runtime.visualize import visualize_scheduler_dag
+from devsper.runtime.executor import Executor
+from devsper.runtime.state_manager import RuntimeStateManager
+from devsper.runtime.task_runner import TaskRunner
+from devsper.runtime.agent_runner import AgentRunner
+from devsper.runtime.tool_runner import ToolRunner
+from devsper.runtime.execution_graph import ExecutionGraph
+from devsper.runtime.planner import RuntimePlanner
+from devsper.runtime.agent_pool import AgentPool
+from devsper.runtime.model_router import ModelRouter
+from devsper.runtime.speculative_planner import SpeculativePlanner
+from devsper.runtime.hitl import HITLManager
+
+__all__ = [
+    "Executor",
+    "RuntimeStateManager",
+    "TaskRunner",
+    "AgentRunner",
+    "ToolRunner",
+    "ExecutionGraph",
+    "RuntimePlanner",
+    "AgentPool",
+    "ModelRouter",
+    "SpeculativePlanner",
+    "HITLManager",
+    "replay_execution",
+    "collect_telemetry",
+    "print_telemetry_summary",
+    "visualize_scheduler_dag",
+]

devsper-2.4.0/devsper/runtime/agent_pool.py

@@ -0,0 +1,85 @@
+from __future__ import annotations
+
+import asyncio
+from collections import defaultdict
+from contextlib import asynccontextmanager
+from dataclasses import dataclass
+from typing import Callable
+
+from devsper.agents.agent import Agent
+from devsper.runtime.agent_runner import AgentRunner
+from devsper.types.task import Task
+
+
+@dataclass(frozen=True)
+class AgentLease:
+    worker_id: str
+    agent: Agent
+
+
+class AgentPool:
+    """Reusable async pool for worker-local agent execution."""
+
+    def __init__(
+        self,
+        agent_factory: Callable[[], Agent],
+        max_agents: int = 8,
+        streaming_tools: bool = False,
+    ) -> None:
+        self._factory = agent_factory
+        self._max_agents = max(1, int(max_agents))
+        self._streaming_tools = bool(streaming_tools)
+        self._queues: dict[str, asyncio.Queue[Agent]] = defaultdict(asyncio.Queue)
+        self._created: dict[str, int] = defaultdict(int)
+        self._lock = asyncio.Lock()
+
+    async def acquire_agent(self, worker_id: str = "default") -> AgentLease:
+        queue = self._queues[worker_id]
+        try:
+            agent = queue.get_nowait()
+            return AgentLease(worker_id=worker_id, agent=agent)
+        except asyncio.QueueEmpty:
+            pass
+
+        async with self._lock:
+            if self._created[worker_id] < self._max_agents:
+                self._created[worker_id] += 1
+                return AgentLease(worker_id=worker_id, agent=self._factory())
+
+        agent = await queue.get()
+        return AgentLease(worker_id=worker_id, agent=agent)
+
+    async def release_agent(self, lease: AgentLease) -> None:
+        await self._queues[lease.worker_id].put(lease.agent)
+
+    async def run_agent(
+        self,
+        task: Task,
+        worker_id: str = "default",
+        model_override: str | None = None,
+    ) -> str:
+        lease = await self.acquire_agent(worker_id=worker_id)
+        try:
+            runner = AgentRunner(lease.agent, streaming_tools=self._streaming_tools)
+            return await runner.run_task(task, model_override=model_override)
+        finally:
+            await self.release_agent(lease)
+
+    async def run_parallel(
+        self,
+        tasks: list[Task],
+        worker_id: str = "default",
+        model_override: str | None = None,
+    ) -> list[str]:
+        return await asyncio.gather(
+            *(self.run_agent(t, worker_id=worker_id, model_override=model_override) for t in tasks)
+        )
+
+    @asynccontextmanager
+    async def lease(self, worker_id: str = "default"):
+        lease = await self.acquire_agent(worker_id=worker_id)
+        try:
+            yield lease.agent
+        finally:
+            await self.release_agent(lease)
+

devsper-2.4.0/devsper/runtime/agent_runner.py

@@ -0,0 +1,79 @@
+from __future__ import annotations
+
+import asyncio
+
+from devsper.agents.agent import Agent
+from devsper.types.task import Task
+from devsper.utils.models import generate
+from devsper.tools.tool_runner import run_tool
+
+
+def _extract_tool_calls(text: str) -> list[tuple[str, dict]]:
+    try:
+        from devsper.agents.agent import _parse_all_tool_calls  # type: ignore
+
+        return _parse_all_tool_calls(text)
+    except Exception:
+        return []
+
+
+class AgentRunner:
+    """Async wrapper for agent execution with optional streaming tool handling."""
+
+    def __init__(self, agent: Agent, streaming_tools: bool = False) -> None:
+        self._agent = agent
+        self._streaming_tools = bool(streaming_tools)
+
+    async def run_task(self, task: Task, model_override: str | None = None) -> str:
+        if self._streaming_tools:
+            try:
+                return await asyncio.to_thread(
+                    self._run_task_streaming_tools_sync,
+                    task,
+                    model_override,
+                )
+            except Exception:
+                pass
+        return await asyncio.to_thread(
+            self._agent.run_task,
+            task,
+            model_override,
+            None,
+        )
+
+    def _run_task_streaming_tools_sync(
+        self,
+        task: Task,
+        model_override: str | None = None,
+    ) -> str:
+        request = self._agent.build_request(task, model_override=model_override)
+        if not getattr(self._agent, "use_tools", False) or not request.tools:
+            return self._agent.run_task(task, model_override=model_override, prefetch_result=None)
+
+        from devsper.agents.agent import PROMPT_TEMPLATE_WITH_TOOLS, _format_tools_section, _get_tools_by_names
+
+        tools = _get_tools_by_names(request.tools)
+        prompt = PROMPT_TEMPLATE_WITH_TOOLS.format(
+            role_prefix=request.system_prompt,
+            task_description=request.task.description,
+            memory_section=request.memory_context or "",
+            message_bus_section="",
+            tools_section=_format_tools_section(tools),
+        )
+        conversation = [prompt]
+        task_type = getattr(task, "role", None) or "general"
+
+        for _ in range(max(1, int(getattr(self._agent, "max_tool_iterations", 5)))):
+            streamed = generate(request.model, "\n\n".join(conversation), stream=True)
+            text = "".join(streamed) if hasattr(streamed, "__iter__") else str(streamed)
+            calls = _extract_tool_calls(text)
+            if not calls:
+                return (text or "").strip()
+
+            conversation.append(f"Response:\n{text}")
+            for name, args in calls:
+                result = run_tool(name, args, task_type=task_type)
+                conversation.append(f"Tool result ({name}):\n{result or ''}")
+
+        return "Max tool iterations reached."
+

devsper-2.4.0/devsper/runtime/event_stream.py

@@ -0,0 +1,58 @@
+from __future__ import annotations
+
+import asyncio
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from typing import Any, AsyncIterator
+
+from devsper.types.event import Event, events
+from devsper.utils.event_logger import EventLog
+
+
+@dataclass
+class RuntimeEvent:
+    type: events
+    payload: dict[str, Any]
+    at: datetime
+
+
+class RuntimeEventStream:
+    """In-process async event stream backed by EventLog."""
+
+    def __init__(self, event_log: EventLog, max_queue_size: int = 2048) -> None:
+        self._event_log = event_log
+        self._queue: asyncio.Queue[RuntimeEvent] = asyncio.Queue(maxsize=max(1, int(max_queue_size)))
+        self._closed = False
+        self._dropped = 0
+
+    async def publish(self, event_type: events, payload: dict[str, Any]) -> None:
+        if self._closed:
+            return
+        now = datetime.now(timezone.utc)
+        self._event_log.append_event(Event(timestamp=now, type=event_type, payload=payload))
+        evt = RuntimeEvent(type=event_type, payload=payload, at=now)
+        try:
+            self._queue.put_nowait(evt)
+        except asyncio.QueueFull:
+            # Backpressure policy: drop oldest event and keep newest.
+            try:
+                _ = self._queue.get_nowait()
+                self._dropped += 1
+            except asyncio.QueueEmpty:
+                pass
+            try:
+                self._queue.put_nowait(evt)
+            except asyncio.QueueFull:
+                self._dropped += 1
+
+    async def subscribe(self) -> AsyncIterator[RuntimeEvent]:
+        while not self._closed:
+            yield await self._queue.get()
+
+    def close(self) -> None:
+        self._closed = True
+
+    @property
+    def dropped_events(self) -> int:
+        return self._dropped
+