pyoco 0.1.0__tar.gz → 0.3.0__tar.gz

pyoco-0.3.0/PKG-INFO

@@ -0,0 +1,146 @@
+Metadata-Version: 2.4
+Name: pyoco
+Version: 0.3.0
+Summary: A workflow engine with sugar syntax
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: pyyaml>=6.0.3
+Requires-Dist: fastapi>=0.100.0
+Requires-Dist: uvicorn>=0.20.0
+Requires-Dist: httpx>=0.24.0
+
+# 🐇 Pyoco
+
+**pyoco is a minimal, pure-Python DAG engine for defining and running simple task-based workflows.**
+
+## Overview
+
+Pyoco is designed to be significantly smaller, lighter, and have fewer dependencies than full-scale workflow engines like Airflow. It is optimized for local development and single-machine execution.
+
+You can define tasks and their dependencies entirely in Python code using decorators and a simple API. There is no need for complex configuration files or external databases.
+
+It is ideal for small jobs, development environments, and personal projects where a full-stack workflow engine would be overkill.
+
+## ✨ Features
+
+- **Pure Python**: No external services or heavy dependencies required.
+- **Minimal DAG model**: Tasks and dependencies are defined directly in code.
+- **Task-oriented**: Focus on "small workflows" that should be easy to read and maintain.
+- **Friendly trace logs**: Runs can be traced step by step from the terminal with cute (or plain) logs.
+- **Parallel Execution**: Automatically runs independent tasks in parallel.
+- **Artifact Management**: Easily save and manage task outputs and files.
+- **Observability**: Track execution with unique Run IDs and detailed state transitions.
+- **Control**: Cancel running workflows gracefully with `Ctrl+C`.
+
+## 📦 Installation
+
+```bash
+pip install pyoco
+```
+
+## 🚀 Usage
+
+Here is a minimal example of a pure-Python workflow.
+
+```python
+from pyoco import task
+from pyoco.core.models import Flow
+from pyoco.core.engine import Engine
+
+@task
+def fetch_data(ctx):
+    print("🐰 Fetching data...")
+    return {"id": 1, "value": "carrot"}
+
+@task
+def process_data(ctx, data):
+    print(f"🥕 Processing: {data['value']}")
+    return data['value'].upper()
+
+@task
+def save_result(ctx, result):
+    print(f"✨ Saved: {result}")
+
+# Define the flow
+flow = Flow(name="hello_pyoco")
+flow >> fetch_data >> process_data >> save_result
+
+# Wire inputs (explicitly for this example)
+process_data.task.inputs = {"data": "$node.fetch_data.output"}
+save_result.task.inputs = {"result": "$node.process_data.output"}
+
+if __name__ == "__main__":
+    engine = Engine()
+    engine.run(flow)
+```
+
+Run it:
+
+```bash
+python examples/hello_pyoco.py
+```
+
+Output:
+
+```
+🐇 pyoco > start flow=hello_pyoco
+🏃 start node=fetch_data
+🐰 Fetching data...
+✅ done node=fetch_data (0.30 ms)
+🏃 start node=process_data
+🥕 Processing: carrot
+✅ done node=process_data (0.23 ms)
+🏃 start node=save_result
+✨ Saved: CARROT
+✅ done node=save_result (0.30 ms)
+🥕 done flow=hello_pyoco
+```
+
+See [examples/hello_pyoco.py](examples/hello_pyoco.py) for the full code.
+
+## 🏗️ Architecture
+
+Pyoco is designed with a simple flow:
+
+```
++-----------+        +------------------+        +-----------------+
+| User Code |  --->  | pyoco.core.Flow  |  --->  | trace/logger    |
+| (Tasks)   |        | (Engine)         |        | (Console/File)  |
++-----------+        +------------------+        +-----------------+
+```
+
+1. **User Code**: You define tasks and flows using Python decorators.
+2. **Core Engine**: The engine resolves dependencies and executes tasks (in parallel where possible).
+3. **Trace**: Execution events are sent to the trace backend for logging (cute or plain).
+
+## 🎭 Modes
+
+Pyoco has two output modes:
+
+- **Cute Mode** (Default): Uses emojis and friendly messages. Best for local development and learning.
+- **Non-Cute Mode**: Plain text logs. Best for CI/CD and production monitoring.
+
+You can switch modes using an environment variable:
+
+```bash
+export PYOCO_CUTE=0  # Disable cute mode
+```
+
+Or via CLI flag:
+
+```bash
+pyoco run --non-cute ...
+```
+
+## 📚 Documentation
+
+- [Tutorials](docs/tutorial/index.md)
+- [Roadmap](docs/roadmap.md)
+
+## 💖 Contributing
+
+We love contributions! Please feel free to submit a Pull Request.
+
+---
+
+*Made with 🥕 by the Pyoco Team.*

pyoco-0.3.0/README.md

@@ -0,0 +1,135 @@
+# 🐇 Pyoco
+
+**pyoco is a minimal, pure-Python DAG engine for defining and running simple task-based workflows.**
+
+## Overview
+
+Pyoco is designed to be significantly smaller, lighter, and have fewer dependencies than full-scale workflow engines like Airflow. It is optimized for local development and single-machine execution.
+
+You can define tasks and their dependencies entirely in Python code using decorators and a simple API. There is no need for complex configuration files or external databases.
+
+It is ideal for small jobs, development environments, and personal projects where a full-stack workflow engine would be overkill.
+
+## ✨ Features
+
+- **Pure Python**: No external services or heavy dependencies required.
+- **Minimal DAG model**: Tasks and dependencies are defined directly in code.
+- **Task-oriented**: Focus on "small workflows" that should be easy to read and maintain.
+- **Friendly trace logs**: Runs can be traced step by step from the terminal with cute (or plain) logs.
+- **Parallel Execution**: Automatically runs independent tasks in parallel.
+- **Artifact Management**: Easily save and manage task outputs and files.
+- **Observability**: Track execution with unique Run IDs and detailed state transitions.
+- **Control**: Cancel running workflows gracefully with `Ctrl+C`.
+
+## 📦 Installation
+
+```bash
+pip install pyoco
+```
+
+## 🚀 Usage
+
+Here is a minimal example of a pure-Python workflow.
+
+```python
+from pyoco import task
+from pyoco.core.models import Flow
+from pyoco.core.engine import Engine
+
+@task
+def fetch_data(ctx):
+    print("🐰 Fetching data...")
+    return {"id": 1, "value": "carrot"}
+
+@task
+def process_data(ctx, data):
+    print(f"🥕 Processing: {data['value']}")
+    return data['value'].upper()
+
+@task
+def save_result(ctx, result):
+    print(f"✨ Saved: {result}")
+
+# Define the flow
+flow = Flow(name="hello_pyoco")
+flow >> fetch_data >> process_data >> save_result
+
+# Wire inputs (explicitly for this example)
+process_data.task.inputs = {"data": "$node.fetch_data.output"}
+save_result.task.inputs = {"result": "$node.process_data.output"}
+
+if __name__ == "__main__":
+    engine = Engine()
+    engine.run(flow)
+```
+
+Run it:
+
+```bash
+python examples/hello_pyoco.py
+```
+
+Output:
+
+```
+🐇 pyoco > start flow=hello_pyoco
+🏃 start node=fetch_data
+🐰 Fetching data...
+✅ done node=fetch_data (0.30 ms)
+🏃 start node=process_data
+🥕 Processing: carrot
+✅ done node=process_data (0.23 ms)
+🏃 start node=save_result
+✨ Saved: CARROT
+✅ done node=save_result (0.30 ms)
+🥕 done flow=hello_pyoco
+```
+
+See [examples/hello_pyoco.py](examples/hello_pyoco.py) for the full code.
+
+## 🏗️ Architecture
+
+Pyoco is designed with a simple flow:
+
+```
++-----------+        +------------------+        +-----------------+
+| User Code |  --->  | pyoco.core.Flow  |  --->  | trace/logger    |
+| (Tasks)   |        | (Engine)         |        | (Console/File)  |
++-----------+        +------------------+        +-----------------+
+```
+
+1. **User Code**: You define tasks and flows using Python decorators.
+2. **Core Engine**: The engine resolves dependencies and executes tasks (in parallel where possible).
+3. **Trace**: Execution events are sent to the trace backend for logging (cute or plain).
+
+## 🎭 Modes
+
+Pyoco has two output modes:
+
+- **Cute Mode** (Default): Uses emojis and friendly messages. Best for local development and learning.
+- **Non-Cute Mode**: Plain text logs. Best for CI/CD and production monitoring.
+
+You can switch modes using an environment variable:
+
+```bash
+export PYOCO_CUTE=0  # Disable cute mode
+```
+
+Or via CLI flag:
+
+```bash
+pyoco run --non-cute ...
+```
+
+## 📚 Documentation
+
+- [Tutorials](docs/tutorial/index.md)
+- [Roadmap](docs/roadmap.md)
+
+## 💖 Contributing
+
+We love contributions! Please feel free to submit a Pull Request.
+
+---
+
+*Made with 🥕 by the Pyoco Team.*

{pyoco-0.1.0 → pyoco-0.3.0}/pyproject.toml

@@ -1,9 +1,12 @@
 [project]
 name = "pyoco"
-version = "0.1.0"
+version = "0.3.0"
 description = "A workflow engine with sugar syntax"
 readme = "README.md"
 requires-python = ">=3.10"
 dependencies = [
     "pyyaml>=6.0.3",
+    "fastapi>=0.100.0",
+    "uvicorn>=0.20.0",
+    "httpx>=0.24.0",
 ]

{pyoco-0.1.0 → pyoco-0.3.0}/src/pyoco/cli/main.py

@@ -1,11 +1,13 @@
 import argparse
 import sys
 import os
+import signal
 from ..schemas.config import PyocoConfig
 from ..discovery.loader import TaskLoader
 from ..core.models import Flow
 from ..core.engine import Engine
 from ..trace.console import ConsoleTraceBackend
+from ..client import Client
 
 def main():
     parser = argparse.ArgumentParser(description="Pyoco Workflow Engine")
@@ -20,6 +22,7 @@ def main():
     run_parser.add_argument("--non-cute", action="store_false", dest="cute", help="Use plain trace style")
     # Allow overriding params via CLI
     run_parser.add_argument("--param", action="append", help="Override params (key=value)")
+    run_parser.add_argument("--server", help="Server URL for remote execution")
 
     # Check command
     check_parser = subparsers.add_parser("check", help="Verify a workflow")
@@ -30,35 +33,136 @@ def main():
     list_parser = subparsers.add_parser("list-tasks", help="List available tasks")
     list_parser.add_argument("--config", required=True, help="Path to flow.yaml")
 
+    # Server command
+    server_parser = subparsers.add_parser("server", help="Manage Kanban Server")
+    server_subparsers = server_parser.add_subparsers(dest="server_command")
+    server_start = server_subparsers.add_parser("start", help="Start the server")
+    server_start.add_argument("--host", default="0.0.0.0", help="Host to bind")
+    server_start.add_argument("--port", type=int, default=8000, help="Port to bind")
+
+    # Worker command
+    worker_parser = subparsers.add_parser("worker", help="Manage Worker")
+    worker_subparsers = worker_parser.add_subparsers(dest="worker_command")
+    worker_start = worker_subparsers.add_parser("start", help="Start a worker")
+    worker_start.add_argument("--server", required=True, help="Server URL")
+    worker_start.add_argument("--config", required=True, help="Path to flow.yaml")
+    worker_start.add_argument("--tags", help="Comma-separated tags")
+
+    # Runs command
+    runs_parser = subparsers.add_parser("runs", help="Manage runs")
+    runs_subparsers = runs_parser.add_subparsers(dest="runs_command")
+    
+    runs_list = runs_subparsers.add_parser("list", help="List runs")
+    runs_list.add_argument("--server", default="http://localhost:8000", help="Server URL")
+    runs_list.add_argument("--status", help="Filter by status")
+    
+    runs_show = runs_subparsers.add_parser("show", help="Show run details")
+    runs_show.add_argument("run_id", help="Run ID")
+    runs_show.add_argument("--server", default="http://localhost:8000", help="Server URL")
+    
+    runs_cancel = runs_subparsers.add_parser("cancel", help="Cancel a run")
+    runs_cancel.add_argument("run_id", help="Run ID")
+    runs_cancel.add_argument("--server", default="http://localhost:8000", help="Server URL")
+
     args = parser.parse_args()
 
     if not args.command:
         parser.print_help()
         sys.exit(1)
 
-    # Load config
-    try:
-        config = PyocoConfig.from_yaml(args.config)
-    except Exception as e:
-        print(f"Error loading config: {e}")
-        sys.exit(1)
+    # Load config only if needed
+    config = None
+    if hasattr(args, 'config') and args.config:
+        try:
+            config = PyocoConfig.from_yaml(args.config)
+        except Exception as e:
+            print(f"Error loading config: {e}")
+            sys.exit(1)
 
-    # Discover tasks
-    loader = TaskLoader(config)
-    loader.load()
+    # Discover tasks only if config is loaded
+    loader = None
+    if config:
+        loader = TaskLoader(config)
+        loader.load()
 
     if args.command == "list-tasks":
+        if not loader:
+             print("Error: Config not loaded.")
+             sys.exit(1)
         print("Available tasks:")
         for name in loader.tasks:
             print(f" - {name}")
         return
 
+    if args.command == "server":
+        if args.server_command == "start":
+            import uvicorn
+            print(f"🐇 Starting Kanban Server on {args.host}:{args.port}")
+            uvicorn.run("pyoco.server.api:app", host=args.host, port=args.port, log_level="info")
+        return
+
+    if args.command == "worker":
+        if args.worker_command == "start":
+            from ..worker.runner import Worker
+            tags = args.tags.split(",") if args.tags else []
+            worker = Worker(args.server, config, tags)
+            worker.start()
+        return
+
+    if args.command == "runs":
+        client = Client(args.server)
+        try:
+            if args.runs_command == "list":
+                runs = client.list_runs(status=args.status)
+                print(f"🐇 Active Runs ({len(runs)}):")
+                print(f"{'ID':<36} | {'Status':<12} | {'Flow':<15}")
+                print("-" * 70)
+                for r in runs:
+                    # RunContext doesn't have flow_name in core model, but store adds it.
+                    # We need to access it safely.
+                    flow_name = r.get("flow_name", "???")
+                    print(f"{r['run_id']:<36} | {r['status']:<12} | {flow_name:<15}")
+            
+            elif args.runs_command == "show":
+                run = client.get_run(args.run_id)
+                print(f"🐇 Run: {run['run_id']}")
+                print(f"Status: {run['status']}")
+                print("Tasks:")
+                for t_name, t_state in run.get("tasks", {}).items():
+                    print(f"  [{t_state}] {t_name}")
+            
+            elif args.runs_command == "cancel":
+                client.cancel_run(args.run_id)
+                print(f"🛑 Cancellation requested for run {args.run_id}")
+        except Exception as e:
+            print(f"Error: {e}")
+        return
+
     if args.command == "run":
         flow_conf = config.flows.get(args.flow)
         if not flow_conf:
             print(f"Flow '{args.flow}' not found in config.")
             sys.exit(1)
         
+        # Params
+        params = flow_conf.defaults.copy()
+        if args.param:
+            for p in args.param:
+                if "=" in p:
+                    k, v = p.split("=", 1)
+                    params[k] = v # Simple string parsing for now
+
+        if args.server:
+            # Remote execution
+            client = Client(args.server)
+            try:
+                run_id = client.submit_run(args.flow, params)
+                print(f"🚀 Flow submitted! Run ID: {run_id}")
+                print(f"📋 View status: pyoco runs show {run_id} --server {args.server}")
+            except Exception as e:
+                print(f"Error submitting flow: {e}")
+                sys.exit(1)
+            return
         # Build Flow from graph string
         from ..dsl.syntax import TaskWrapper
         eval_context = {name: TaskWrapper(task) for name, task in loader.tasks.items()}
@@ -76,13 +180,15 @@ def main():
             backend = ConsoleTraceBackend(style="cute" if args.cute else "plain")
             engine = Engine(trace_backend=backend)
             
-            # Params
-            params = flow_conf.defaults.copy()
-            if args.param:
-                for p in args.param:
-                    if "=" in p:
-                        k, v = p.split("=", 1)
-                        params[k] = v # Simple string parsing for now
+            # Params (Moved up)
+            
+            # Signal handler for cancellation
+            def signal_handler(sig, frame):
+                print("\n🛑 Ctrl+C detected. Cancelling active runs...")
+                for rid in list(engine.active_runs.keys()):
+                    engine.cancel(rid)
+            
+            signal.signal(signal.SIGINT, signal_handler)
             
             engine.run(flow, params)
             

pyoco-0.3.0/src/pyoco/client.py

@@ -0,0 +1,69 @@
+import httpx
+from typing import Dict, List, Optional, Any
+from .core.models import RunStatus, TaskState
+
+class Client:
+    def __init__(self, server_url: str, client_id: str = "cli"):
+        self.server_url = server_url.rstrip("/")
+        self.client_id = client_id
+        self.client = httpx.Client(base_url=self.server_url)
+
+    def submit_run(self, flow_name: str, params: Dict[str, Any], tags: List[str] = []) -> str:
+        resp = self.client.post("/runs", json={
+            "flow_name": flow_name,
+            "params": params,
+            "tags": tags
+        })
+        resp.raise_for_status()
+        return resp.json()["run_id"]
+
+    def list_runs(self, status: Optional[str] = None) -> List[Dict]:
+        params = {}
+        if status:
+            params["status"] = status
+        resp = self.client.get("/runs", params=params)
+        resp.raise_for_status()
+        return resp.json()
+
+    def get_run(self, run_id: str) -> Dict:
+        resp = self.client.get(f"/runs/{run_id}")
+        resp.raise_for_status()
+        return resp.json()
+
+    def cancel_run(self, run_id: str):
+        resp = self.client.post(f"/runs/{run_id}/cancel")
+        resp.raise_for_status()
+
+    def poll(self, tags: List[str] = []) -> Optional[Dict[str, Any]]:
+        try:
+            resp = self.client.post("/workers/poll", json={
+                "worker_id": self.client_id,
+                "tags": tags
+            })
+            resp.raise_for_status()
+            data = resp.json()
+            if data.get("run_id"):
+                return data
+            return None
+        except Exception as e:
+            # print(f"Poll failed: {e}")
+            return None
+
+    def heartbeat(self, run_id: str, task_states: Dict[str, TaskState], run_status: RunStatus) -> bool:
+        """
+        Sends heartbeat. Returns True if cancellation is requested.
+        """
+        try:
+            # Convert Enums to values
+            states_json = {k: v.value if hasattr(v, 'value') else v for k, v in task_states.items()}
+            status_value = run_status.value if hasattr(run_status, 'value') else run_status
+            
+            resp = self.client.post(f"/runs/{run_id}/heartbeat", json={
+                "task_states": states_json,
+                "run_status": status_value
+            })
+            resp.raise_for_status()
+            return resp.json().get("cancel_requested", False)
+        except Exception as e:
+            print(f"Heartbeat failed: {e}")
+            return False

{pyoco-0.1.0 → pyoco-0.3.0}/src/pyoco/core/context.py

@@ -1,21 +1,37 @@
 import threading
-from typing import Any, Dict, Optional
+from typing import Any, Dict, List, Optional
 from dataclasses import dataclass, field
+from .models import RunContext
 
 @dataclass
 class Context:
+    """
+    Execution context passed to tasks.
+    """
     params: Dict[str, Any] = field(default_factory=dict)
-    env: Dict[str, str] = field(default_factory=dict)
     results: Dict[str, Any] = field(default_factory=dict)
     scratch: Dict[str, Any] = field(default_factory=dict)
     artifacts: Dict[str, Any] = field(default_factory=dict)
-    run_id: Optional[str] = None
-    artifact_dir: str = field(default="./artifacts")
+    env: Dict[str, str] = field(default_factory=dict)
+    artifact_dir: Optional[str] = None
+    
+    # Reference to the parent run context (v0.2.0+)
+    run_context: Optional[RunContext] = None
     
     _lock: threading.Lock = field(default_factory=threading.Lock, repr=False)
 
+    @property
+    def is_cancelled(self) -> bool:
+        if self.run_context:
+            from .models import RunStatus
+            return self.run_context.status in [RunStatus.CANCELLING, RunStatus.CANCELLED]
+        return False
+
     def __post_init__(self):
         # Ensure artifact directory exists
+        if self.artifact_dir is None:
+            self.artifact_dir = "./artifacts"
+            
         import pathlib
         pathlib.Path(self.artifact_dir).mkdir(parents=True, exist_ok=True)