penguiflow 2.0.0__tar.gz → 2.1.0__tar.gz

{penguiflow-2.0.0 → penguiflow-2.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: penguiflow
-Version: 2.0.0
+Version: 2.1.0
 Summary: Async agent orchestration primitives.
 Author: PenguiFlow Team
 License: MIT License
@@ -37,6 +37,10 @@ Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
 Requires-Dist: pytest-cov>=4.0; extra == "dev"
 Requires-Dist: coverage[toml]>=7.0; extra == "dev"
 Requires-Dist: ruff>=0.2; extra == "dev"
+Requires-Dist: fastapi>=0.110; extra == "dev"
+Requires-Dist: httpx>=0.27; extra == "dev"
+Provides-Extra: a2a-server
+Requires-Dist: fastapi>=0.110; extra == "a2a-server"
 Dynamic: license-file
 
 # PenguiFlow 🐧❄️
@@ -77,6 +81,18 @@ It provides:
 * **Observability hooks** (`FlowEvent` callbacks for logging, MLflow, or custom metrics sinks)
 * **Policy-driven routing** (optional policies steer routers without breaking existing flows)
 * **Traceable exceptions** (`FlowError` captures node/trace metadata and optionally emits to Rookery)
+* **Distribution hooks (opt-in)** — plug a `StateStore` to persist trace history and a
+  `MessageBus` to publish floe traffic for remote workers without changing existing flows.
+* **Remote calls (opt-in)** — `RemoteNode` bridges the runtime to external agents through a
+  pluggable `RemoteTransport` interface (A2A-ready) while propagating streaming chunks and
+  cancellation.
+* **A2A server adapter (opt-in)** — wrap a PenguiFlow graph in a FastAPI surface using
+  `penguiflow_a2a.A2AServerAdapter` so other agents can call `message/send`,
+  `message/stream`, and `tasks/cancel` while reusing the runtime's backpressure and
+  cancellation semantics.
+* **Observability & ops polish** — remote calls emit structured metrics (latency, payload
+  sizes, cancel reasons) and the `penguiflow-admin` CLI replays trace history from any
+  configured `StateStore` for debugging.
 
 Built on pure `asyncio` (no threads), PenguiFlow is small, predictable, and repo-agnostic.
 Product repos only define **their models + node functions** — the core stays dependency-light.
@@ -168,6 +184,10 @@ print(out.payload)            # PackOut(...)
 await flow.stop()
 ```
 
+> **Opt-in distribution:** pass `state_store=` and/or `message_bus=` when calling
+> `penguiflow.core.create(...)` to persist trace history and publish floe traffic
+> without changing node logic.
+
 ---
 
 ## 🧭 Design Principles
@@ -222,6 +242,60 @@ sacrificing backpressure or ordering guarantees.  The helper wraps the payload i
 increments per-stream sequence numbers.  See `tests/test_streaming.py` and
 `examples/streaming_llm/` for an end-to-end walk-through.
 
+### Remote orchestration
+
+Phase 2 introduces `RemoteNode` and the `RemoteTransport` protocol so flows can delegate
+work to remote agents (e.g., the A2A JSON-RPC/SSE ecosystem) without changing existing
+nodes.  The helper records remote bindings via the `StateStore`, mirrors streaming
+partials back into the graph, and propagates per-trace cancellation to remote tasks via
+`RemoteTransport.cancel`.  See `tests/test_remote.py` for reference in-memory transports.
+
+### Exposing a flow over A2A
+
+Install the optional extra to expose PenguiFlow as an A2A-compatible FastAPI service:
+
+```bash
+pip install "penguiflow[a2a-server]"
+```
+
+Create the adapter and mount the routes:
+
+```python
+from penguiflow import Message, Node, create
+from penguiflow_a2a import A2AAgentCard, A2AServerAdapter, A2ASkill, create_a2a_app
+
+async def orchestrate(message: Message, ctx):
+    await ctx.emit_chunk(parent=message, text="thinking...")
+    return {"result": "done"}
+
+node = Node(orchestrate, name="main")
+flow = create(node.to())
+
+card = A2AAgentCard(
+    name="Main Agent",
+    description="Primary entrypoint for orchestration",
+    version="2.1.0",
+    skills=[A2ASkill(name="orchestrate", description="Handles orchestration")],
+)
+
+adapter = A2AServerAdapter(
+    flow,
+    agent_card=card,
+    agent_url="https://agent.example",
+)
+app = create_a2a_app(adapter)
+```
+
+The generated FastAPI app implements:
+
+* `GET /agent` for discovery (Agent Card)
+* `POST /message/send` for unary execution
+* `POST /message/stream` for SSE streaming
+* `POST /tasks/cancel` to mirror cancellation into PenguiFlow traces
+
+`A2AServerAdapter` reuses the runtime's `StateStore` hooks, so bindings between trace IDs
+and external `taskId`/`contextId` pairs are persisted automatically.
+
 ### Reliability & guardrails
 
 PenguiFlow enforces reliability boundaries out of the box:
@@ -478,9 +552,15 @@ docs or diagramming pipelines.
 * **Structured `FlowEvent`s**: every node event carries `{ts, trace_id, node_name, event,
   latency_ms, q_depth_in, q_depth_out, attempt}` plus a mutable `extra` map for custom
   annotations.
+* **Remote call telemetry**: `RemoteNode` executions emit extra metrics (latency, request
+  and response bytes, context/task identifiers, cancel reasons) so remote hops can be
+  traced end-to-end.
 * **Middleware hooks**: subscribe observers (e.g., MLflow) to the structured `FlowEvent`
   stream. See `examples/mlflow_metrics/` for an MLflow integration and
   `examples/reliability_middleware/` for a concrete timeout + retry walkthrough.
+* **`penguiflow-admin` CLI**: inspect or replay stored trace history from any configured
+  `StateStore` (`penguiflow-admin history <trace>` or `penguiflow-admin replay <trace>`)
+  when debugging distributed runs.
 
 ---
 
@@ -488,9 +568,9 @@ docs or diagramming pipelines.
 
 - **In-process runtime**: there is no built-in distribution layer yet. Long-running CPU work should be delegated to your own pools or services.
 - **Registry-driven typing**: nodes default to validation. Provide a `ModelRegistry` when calling `flow.run(...)` or set `validate="none"` explicitly for untyped hops.
-- **Observability**: structured `FlowEvent` callbacks power logs/metrics; integrations with
-  third-party stacks (OTel, Prometheus, Datadog) remain DIY. See the MLflow middleware
-  example for a lightweight pattern.
+- **Observability**: structured `FlowEvent` callbacks and the `penguiflow-admin` CLI power
+  local debugging; integrations with third-party stacks (OTel, Prometheus, Datadog) remain
+  DIY. See the MLflow middleware example for a lightweight pattern.
 - **Roadmap**: follow-up releases focus on optional distributed backends, deeper observability integrations, and additional playbook patterns. Contributions and proposals are welcome!
 
 ---

{penguiflow-2.0.0 → penguiflow-2.1.0}/README.md

@@ -36,6 +36,18 @@ It provides:
 * **Observability hooks** (`FlowEvent` callbacks for logging, MLflow, or custom metrics sinks)
 * **Policy-driven routing** (optional policies steer routers without breaking existing flows)
 * **Traceable exceptions** (`FlowError` captures node/trace metadata and optionally emits to Rookery)
+* **Distribution hooks (opt-in)** — plug a `StateStore` to persist trace history and a
+  `MessageBus` to publish floe traffic for remote workers without changing existing flows.
+* **Remote calls (opt-in)** — `RemoteNode` bridges the runtime to external agents through a
+  pluggable `RemoteTransport` interface (A2A-ready) while propagating streaming chunks and
+  cancellation.
+* **A2A server adapter (opt-in)** — wrap a PenguiFlow graph in a FastAPI surface using
+  `penguiflow_a2a.A2AServerAdapter` so other agents can call `message/send`,
+  `message/stream`, and `tasks/cancel` while reusing the runtime's backpressure and
+  cancellation semantics.
+* **Observability & ops polish** — remote calls emit structured metrics (latency, payload
+  sizes, cancel reasons) and the `penguiflow-admin` CLI replays trace history from any
+  configured `StateStore` for debugging.
 
 Built on pure `asyncio` (no threads), PenguiFlow is small, predictable, and repo-agnostic.
 Product repos only define **their models + node functions** — the core stays dependency-light.
@@ -127,6 +139,10 @@ print(out.payload)            # PackOut(...)
 await flow.stop()
 ```
 
+> **Opt-in distribution:** pass `state_store=` and/or `message_bus=` when calling
+> `penguiflow.core.create(...)` to persist trace history and publish floe traffic
+> without changing node logic.
+
 ---
 
 ## 🧭 Design Principles
@@ -181,6 +197,60 @@ sacrificing backpressure or ordering guarantees.  The helper wraps the payload i
 increments per-stream sequence numbers.  See `tests/test_streaming.py` and
 `examples/streaming_llm/` for an end-to-end walk-through.
 
+### Remote orchestration
+
+Phase 2 introduces `RemoteNode` and the `RemoteTransport` protocol so flows can delegate
+work to remote agents (e.g., the A2A JSON-RPC/SSE ecosystem) without changing existing
+nodes.  The helper records remote bindings via the `StateStore`, mirrors streaming
+partials back into the graph, and propagates per-trace cancellation to remote tasks via
+`RemoteTransport.cancel`.  See `tests/test_remote.py` for reference in-memory transports.
+
+### Exposing a flow over A2A
+
+Install the optional extra to expose PenguiFlow as an A2A-compatible FastAPI service:
+
+```bash
+pip install "penguiflow[a2a-server]"
+```
+
+Create the adapter and mount the routes:
+
+```python
+from penguiflow import Message, Node, create
+from penguiflow_a2a import A2AAgentCard, A2AServerAdapter, A2ASkill, create_a2a_app
+
+async def orchestrate(message: Message, ctx):
+    await ctx.emit_chunk(parent=message, text="thinking...")
+    return {"result": "done"}
+
+node = Node(orchestrate, name="main")
+flow = create(node.to())
+
+card = A2AAgentCard(
+    name="Main Agent",
+    description="Primary entrypoint for orchestration",
+    version="2.1.0",
+    skills=[A2ASkill(name="orchestrate", description="Handles orchestration")],
+)
+
+adapter = A2AServerAdapter(
+    flow,
+    agent_card=card,
+    agent_url="https://agent.example",
+)
+app = create_a2a_app(adapter)
+```
+
+The generated FastAPI app implements:
+
+* `GET /agent` for discovery (Agent Card)
+* `POST /message/send` for unary execution
+* `POST /message/stream` for SSE streaming
+* `POST /tasks/cancel` to mirror cancellation into PenguiFlow traces
+
+`A2AServerAdapter` reuses the runtime's `StateStore` hooks, so bindings between trace IDs
+and external `taskId`/`contextId` pairs are persisted automatically.
+
 ### Reliability & guardrails
 
 PenguiFlow enforces reliability boundaries out of the box:
@@ -437,9 +507,15 @@ docs or diagramming pipelines.
 * **Structured `FlowEvent`s**: every node event carries `{ts, trace_id, node_name, event,
   latency_ms, q_depth_in, q_depth_out, attempt}` plus a mutable `extra` map for custom
   annotations.
+* **Remote call telemetry**: `RemoteNode` executions emit extra metrics (latency, request
+  and response bytes, context/task identifiers, cancel reasons) so remote hops can be
+  traced end-to-end.
 * **Middleware hooks**: subscribe observers (e.g., MLflow) to the structured `FlowEvent`
   stream. See `examples/mlflow_metrics/` for an MLflow integration and
   `examples/reliability_middleware/` for a concrete timeout + retry walkthrough.
+* **`penguiflow-admin` CLI**: inspect or replay stored trace history from any configured
+  `StateStore` (`penguiflow-admin history <trace>` or `penguiflow-admin replay <trace>`)
+  when debugging distributed runs.
 
 ---
 
@@ -447,9 +523,9 @@ docs or diagramming pipelines.
 
 - **In-process runtime**: there is no built-in distribution layer yet. Long-running CPU work should be delegated to your own pools or services.
 - **Registry-driven typing**: nodes default to validation. Provide a `ModelRegistry` when calling `flow.run(...)` or set `validate="none"` explicitly for untyped hops.
-- **Observability**: structured `FlowEvent` callbacks power logs/metrics; integrations with
-  third-party stacks (OTel, Prometheus, Datadog) remain DIY. See the MLflow middleware
-  example for a lightweight pattern.
+- **Observability**: structured `FlowEvent` callbacks and the `penguiflow-admin` CLI power
+  local debugging; integrations with third-party stacks (OTel, Prometheus, Datadog) remain
+  DIY. See the MLflow middleware example for a lightweight pattern.
 - **Roadmap**: follow-up releases focus on optional distributed backends, deeper observability integrations, and additional playbook patterns. Contributions and proposals are welcome!
 
 ---

{penguiflow-2.0.0 → penguiflow-2.1.0}/penguiflow/__init__.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 from . import testkit
+from .bus import BusEnvelope, MessageBus
 from .core import (
     DEFAULT_QUEUE_MAXSIZE,
     Context,
@@ -18,6 +19,14 @@ from .node import Node, NodePolicy
 from .patterns import join_k, map_concurrent, predicate_router, union_router
 from .policies import DictRoutingPolicy, RoutingPolicy, RoutingRequest
 from .registry import ModelRegistry
+from .remote import (
+    RemoteCallRequest,
+    RemoteCallResult,
+    RemoteNode,
+    RemoteStreamEvent,
+    RemoteTransport,
+)
+from .state import RemoteBinding, StateStore, StoredEvent
 from .streaming import (
     chunk_to_ws_json,
     emit_stream_events,
@@ -40,6 +49,8 @@ __all__ = [
     "FlowEvent",
     "FlowError",
     "FlowErrorCode",
+    "MessageBus",
+    "BusEnvelope",
     "call_playbook",
     "Headers",
     "Message",
@@ -63,6 +74,14 @@ __all__ = [
     "flow_to_dot",
     "create",
     "testkit",
+    "StateStore",
+    "StoredEvent",
+    "RemoteBinding",
+    "RemoteTransport",
+    "RemoteCallRequest",
+    "RemoteCallResult",
+    "RemoteStreamEvent",
+    "RemoteNode",
 ]
 
-__version__ = "2.0.0"
+__version__ = "2.1.0"

penguiflow-2.1.0/penguiflow/admin.py

@@ -0,0 +1,174 @@
+"""Developer CLI helpers for inspecting PenguiFlow trace history."""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import importlib
+import json
+import sys
+from collections.abc import Callable, Sequence
+from typing import Any
+
+from .state import StateStore, StoredEvent
+
+__all__ = ["load_state_store", "render_events", "main"]
+
+
+class _Args(argparse.Namespace):
+    handler: Callable[[_Args], Any]
+    state_store: str
+    trace_id: str
+    tail: int | None
+    delay: float
+
+
+def _resolve_factory(spec: str) -> Callable[[], Any]:
+    module_name, _, attr = spec.partition(":")
+    if not module_name or not attr:
+        raise ValueError(
+            "state store spec must be in the form 'package.module:callable'"
+        )
+    module = importlib.import_module(module_name)
+    try:
+        factory = getattr(module, attr)
+    except AttributeError as exc:  # pragma: no cover - defensive guard
+        raise ValueError(f"{spec!r} does not resolve to a callable") from exc
+    if not callable(factory):
+        raise TypeError(f"{spec!r} resolved to {type(factory)!r}, not a callable")
+    return factory
+
+
+async def load_state_store(spec: str) -> StateStore:
+    """Instantiate a :class:`StateStore` from ``module:callable`` spec."""
+
+    factory = _resolve_factory(spec)
+    instance = factory()
+    if asyncio.iscoroutine(instance):
+        instance = await instance
+    required = ("save_event", "load_history", "save_remote_binding")
+    if not all(hasattr(instance, attr) for attr in required):  # pragma: no cover
+        raise TypeError(
+            "StateStore factories must implement "
+            "save_event/load_history/save_remote_binding"
+        )
+    return instance
+
+
+def _trim_events(events: Sequence[StoredEvent], tail: int | None) -> list[StoredEvent]:
+    items = list(events)
+    if tail is None:
+        return items
+    if tail <= 0:
+        return []
+    return items[-tail:]
+
+
+def render_events(
+    events: Sequence[StoredEvent], *, tail: int | None = None
+) -> list[str]:
+    """Return JSON line representations of ``events`` (optionally tail-truncated)."""
+
+    trimmed = _trim_events(events, tail)
+    lines: list[str] = []
+    for event in trimmed:
+        payload = dict(event.payload)
+        payload.setdefault("event", event.kind)
+        payload.setdefault("trace_id", event.trace_id)
+        payload.setdefault("node_name", event.node_name)
+        payload.setdefault("node_id", event.node_id)
+        payload.setdefault("ts", event.ts)
+        lines.append(json.dumps(payload, sort_keys=True, default=str))
+    return lines
+
+
+async def _cmd_history(args: _Args) -> None:
+    store = await load_state_store(args.state_store)
+    events = await store.load_history(args.trace_id)
+    for line in render_events(events, tail=args.tail):
+        print(line)
+
+
+async def _cmd_replay(args: _Args) -> None:
+    store = await load_state_store(args.state_store)
+    events = _trim_events(await store.load_history(args.trace_id), args.tail)
+    total = len(events)
+    if not total:
+        print(f"# trace {args.trace_id} has no stored events")
+        return
+    print(f"# replay trace={args.trace_id} events={total}")
+    for event in events:
+        payload = render_events([event])[0]
+        print(payload)
+        if args.delay > 0:
+            await asyncio.sleep(args.delay)
+
+
+def _build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="penguiflow-admin",
+        description=(
+            "Inspect PenguiFlow trace history via configured StateStore "
+            "adapters."
+        ),
+    )
+    common = argparse.ArgumentParser(add_help=False)
+    common.add_argument(
+        "--state-store",
+        required=True,
+        help="Import path to a factory returning a StateStore (module:callable)",
+    )
+    common.add_argument(
+        "--tail",
+        type=int,
+        default=None,
+        help="Only show the last N events from the trace history.",
+    )
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    history = subparsers.add_parser(
+        "history",
+        parents=[common],
+        help="Print stored events for a trace as JSON lines.",
+    )
+    history.add_argument("trace_id", help="Trace identifier to inspect")
+    history.set_defaults(handler=_cmd_history)
+
+    replay = subparsers.add_parser(
+        "replay",
+        parents=[common],
+        help="Replay events with optional delay to mimic runtime emission.",
+    )
+    replay.add_argument("trace_id", help="Trace identifier to replay")
+    replay.add_argument(
+        "--delay",
+        type=float,
+        default=0.0,
+        help="Sleep duration (seconds) between events when replaying.",
+    )
+    replay.set_defaults(handler=_cmd_replay)
+
+    return parser
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    """Entry point for the ``penguiflow-admin`` CLI."""
+
+    parser = _build_parser()
+    args = parser.parse_args(argv)
+    handler = getattr(args, "handler", None)
+    if handler is None:  # pragma: no cover - argparse guard
+        parser.print_help()
+        return 1
+
+    try:
+        asyncio.run(handler(args))
+    except Exception as exc:  # pragma: no cover - runtime guard
+        print(f"error: {exc}", file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":  # pragma: no cover - manual invocation
+    raise SystemExit(main())
+

penguiflow-2.1.0/penguiflow/bus.py

@@ -0,0 +1,30 @@
+"""Message bus protocol for distributed PenguiFlow edges."""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from dataclasses import dataclass
+from typing import Any, Protocol
+
+
+@dataclass(slots=True)
+class BusEnvelope:
+    """Structured payload published to a :class:`MessageBus`."""
+
+    edge: str
+    source: str | None
+    target: str | None
+    trace_id: str | None
+    payload: Any
+    headers: Mapping[str, Any] | None
+    meta: Mapping[str, Any] | None
+
+
+class MessageBus(Protocol):
+    """Protocol for pluggable message bus adapters."""
+
+    async def publish(self, envelope: BusEnvelope) -> None:
+        """Publish an envelope for downstream workers."""
+
+
+__all__ = ["BusEnvelope", "MessageBus"]