quantalogic 0.50.29__py3-none-any.whl → 0.52.0__py3-none-any.whl

quantalogic/flow/flow_yaml.md

@@ -1,419 +1,490 @@
-# Quantalogic Flow YAML DSL Specification
+# Quantalogic Flow YAML DSL Specification 🚀
 
-## 1. Introduction
 
-The Quantalogic Flow YAML DSL (Domain Specific Language) offers a structured and human-readable way to define workflows. As of February 23, 2025, it provides a rich set of features for complex automation, including:
 
-*   **Function Execution**: Executes asynchronous Python functions, either embedded directly or sourced from PyPI packages, local files, or remote URLs.
-*   **Execution Flow**: Supports sequential, conditional, and parallel transitions between nodes.
-*   **Sub-Workflows**: Enables hierarchical workflows through nested sub-workflows.
-*   **LLM Integration**: Incorporates Large Language Model (LLM) nodes with plain text (`llm_node`) or structured output (`structured_llm_node`), using configurable prompts and parameters.
-*   **Context Management**: Maintains state across nodes using a shared context dictionary.
-*   **Robustness**: Provides configurable retries and timeouts for fault-tolerant execution.
-*   **Programmatic Control**: Managed via the `WorkflowManager` class for dynamic creation and execution.
+## 1. Introduction 🌟
 
-This DSL seamlessly integrates with the `Workflow`, `WorkflowEngine`, and `WorkflowManager` classes from the `quantalogic.flow` package, and it leverages the `Nodes` class for LLM functionality to minimize redundancy.
+The **Quantalogic Flow YAML DSL** is a human-readable, declarative language for defining workflows within the `quantalogic.flow` Python package. As of **March 2, 2025**, it empowers developers to automate tasks with a rich feature set:
 
-## 2. Workflow Structure
+- **Function Execution** ⚙️: Run async Python functions from embedded code, PyPI, local files, or URLs.
+- **Execution Flow** ➡️: Support sequential, conditional, and parallel transitions.
+- **Sub-Workflows** 🌳: Enable hierarchical, modular designs.
+- **LLM Integration** 🤖: Harness Large Language Models for text or structured outputs.
+- **Context Management** 📦: Share state dynamically across nodes.
+- **Robustness** 🛡️: Include retries, delays, and timeouts.
+- **Observers** 👀: Monitor execution with custom handlers.
+- **Programmatic Control** 🧑‍💻: Manage workflows via `WorkflowManager`.
 
-A YAML workflow file consists of three main sections:
+This DSL integrates with `Workflow`, `WorkflowEngine`, and `Nodes` classes, making it ideal for everything from simple scripts to AI-driven workflows. To illustrate, we’ll use a **Story Generator Workflow** as a running example, derived from `examples/qflow/story_generator_agent.py`. Let’s dive in! 🎉
 
-*   **`functions`**: Defines Python functions used by nodes, supporting both inline code and external modules.
-*   **`nodes`**: Configures individual tasks, linking to functions, sub-workflows, or LLM setups.
-*   **`workflow`**: Specifies the execution flow, including the start node and transitions.
+```mermaid
+graph TD
+    A[YAML Workflow File] -->|Defines| B[functions ⚙️]
+    A -->|Configures| C[nodes 🧩]
+    A -->|Orchestrates| D[workflow 🌐]
+    style A fill:#f9f9ff,stroke:#333,stroke-width:2px,stroke-dasharray:5
+    style B fill:#e6f3ff,stroke:#0066cc
+    style C fill:#e6ffe6,stroke:#009933
+    style D fill:#fff0e6,stroke:#cc3300
+```
+
+---
+
+## 2. Workflow Structure 🗺️
+
+A workflow YAML file is divided into three core sections:
+
+- **`functions`**: Python code definitions.
+- **`nodes`**: Task specifications.
+- **`workflow`**: Flow orchestration.
+
+Here’s the skeleton:
 
 ```yaml
 functions:
-  # Function definitions
+  # Python magic ✨
 nodes:
-  # Node configurations
+  # Tasks 🎯
 workflow:
-  # Start node and transitions
+  # Flow control 🚦
+observers:
+  # Event watchers 👀 (optional)
 ```
 
-```mermaid
-graph LR
-    A[YAML Workflow File] --> B(functions);
-    A --> C(nodes);
-    A --> D(workflow);
-    style A fill:#f9f,stroke:#333,stroke-width:2px
-```
-
-## 3. Functions
+### Story Generator Example
+Imagine a workflow that generates a multi-chapter story. We’ll build it step-by-step, starting with its Python form (`story_generator_agent.py`), then its YAML equivalent.
 
-The `functions` section maps function names to their implementations, which can be embedded in the YAML or sourced externally from Python modules.
+---
 
-**Fields**
+## 3. Case Study: Story Generator Workflow 📖
 
-*   `type` (string, required): Specifies the function source. Options:
-    *   `"embedded"`: Inline Python code.
-    *   `"external"`: Module-based function.
-*   `code` (string, optional): Multi-line asynchronous Python code for embedded functions. Required if `type: embedded`.
-*   `module` (string, optional): Source of the external module for external functions. Can be:
-    *   A PyPI package name (e.g., `"requests"`, `"numpy"`).
-    *   A local file path (e.g., `"/path/to/module.py"`).
-    *   A remote URL (e.g., `"https://example.com/module.py"`). Required if `type: external`.
-*   `function` (string, optional): Name of the function within the module for external functions. Required if `type: external`.
+### Python Version (`story_generator_agent.py`)
 
-**Rules**
+This script generates a story outline and chapters iteratively:
 
-*   Embedded functions must be asynchronous (using `async def`) and match the dictionary key name.
-*   External functions require both `module` and `function` fields; `code` must not be present.
-*   For PyPI modules, ensure the package is installed in the Python environment (e.g., via `pip install <module>`).
+```python
+#!/usr/bin/env python
+from quantalogic.flow import Nodes, Workflow
+import anyio
+
+MODEL = "gemini/gemini-2.0-flash"
+DEFAULT_LLM_PARAMS = {"model": MODEL, "temperature": 0.7, "max_tokens": 1000}
+
+@Nodes.llm_node(system_prompt="You are a creative writer skilled at generating stories.", 
+                prompt_template="Create a story outline for a {genre} story with {num_chapters} chapters.", 
+                output="outline", **DEFAULT_LLM_PARAMS)
+def generate_outline(genre, num_chapters):
+    return {}
+
+@Nodes.llm_node(system_prompt="You are a creative writer.", 
+                prompt_template="Write chapter {chapter_num} for this story outline: {outline}. Style: {style}.", 
+                output="chapter", **DEFAULT_LLM_PARAMS)
+def generate_chapter(outline, chapter_num, style):
+    return {}
+
+@Nodes.define(output="updated_context")
+async def update_progress(**context):
+    chapters = context.get('chapters', [])
+    completed_chapters = context.get('completed_chapters', 0)
+    chapter = context.get('chapter', '')
+    updated_chapters = chapters + [chapter]
+    return {**context, "chapters": updated_chapters, "completed_chapters": completed_chapters + 1}
+
+@Nodes.define(output="continue_generating")
+async def check_if_complete(completed_chapters=0, num_chapters=0, **kwargs):
+    return completed_chapters < num_chapters
+
+workflow = (
+    Workflow("generate_outline")
+    .then("generate_chapter")
+    .then("update_progress")
+    .then("check_if_complete")
+    .then("generate_chapter", condition=lambda ctx: ctx.get("continue_generating", False))
+    .then("update_progress")
+    .then("check_if_complete")
+)
+
+def story_observer(event_type, data=None):
+    print(f"Event: {event_type} - Data: {data}")
+workflow.add_observer(story_observer)
+
+if __name__ == "__main__":
+    async def main():
+        initial_context = {
+            "genre": "science fiction",
+            "num_chapters": 3,
+            "chapters": [],
+            "completed_chapters": 0,
+            "style": "descriptive"
+        }
+        engine = workflow.build()
+        result = await engine.run(initial_context)
+        print(f"Completed chapters: {result.get('completed_chapters', 0)}")
+    anyio.run(main)
+```
 
-**Examples**
+### YAML Version (`story_generator_workflow.yaml`)
 
-*   **Embedded Function**
+Here’s the equivalent YAML:
 
 ```yaml
 functions:
-  validate_order:
+  generate_outline:
     type: embedded
     code: |
-      async def validate_order(order: dict) -> bool:
-          await asyncio.sleep(1)
-          return bool(order.get("items"))
-```
-
-*   **External Function from PyPI**
-
-```yaml
-functions:
-  fetch_data:
-    type: external
-    module: requests
-    function: get
-```
-
-Note: Requires `pip install requests` if not already installed.
+      async def generate_outline(genre: str, num_chapters: int) -> str:
+          return ""
+  generate_chapter:
+    type: embedded
+    code: |
+      async def generate_chapter(outline: str, chapter_num: int, style: str) -> str:
+          return ""
+  update_progress:
+    type: embedded
+    code: |
+      async def update_progress(**context):
+          chapters = context.get('chapters', [])
+          completed_chapters = context.get('completed_chapters', 0)
+          chapter = context.get('chapter', '')
+          return {**context, "chapters": chapters + [chapter], "completed_chapters": completed_chapters + 1}
+  check_if_complete:
+    type: embedded
+    code: |
+      async def check_if_complete(completed_chapters=0, num_chapters=0, **kwargs):
+          return completed_chapters < num_chapters
+  story_observer:
+    type: embedded
+    code: |
+      def story_observer(event_type, data=None):
+          print(f"Event: {event_type} - Data: {data}")
 
-*   **External Function from Local File**
+nodes:
+  generate_outline:
+    llm_config:
+      model: "gemini/gemini-2.0-flash"
+      system_prompt: "You are a creative writer skilled at generating stories."
+      prompt_template: "Create a story outline for a {genre} story with {num_chapters} chapters."
+      temperature: 0.7
+      max_tokens: 1000
+    output: outline
+  generate_chapter:
+    llm_config:
+      model: "gemini/gemini-2.0-flash"
+      system_prompt: "You are a creative writer."
+      prompt_template: "Write chapter {chapter_num} for this story outline: {outline}. Style: {style}."
+      temperature: 0.7
+      max_tokens: 1000
+    output: chapter
+  update_progress:
+    function: update_progress
+    output: updated_context
+  check_if_complete:
+    function: check_if_complete
+    output: continue_generating
 
-```yaml
-functions:
-  process_data:
-    type: external
-    module: /path/to/my_module.py
-    function: process
+workflow:
+  start: generate_outline
+  transitions:
+    - from_node: generate_outline
+      to_node: generate_chapter
+    - from_node: generate_chapter
+      to_node: update_progress
+    - from_node: update_progress
+      to_node: check_if_complete
+    - from_node: check_if_complete
+      to_node: generate_chapter
+      condition: "ctx['continue_generating']"
+
+observers:
+  - story_observer
 ```
 
-*   **External Function from URL**
-
-```yaml
-functions:
-  analyze:
-    type: external
-    module: https://example.com/analyze.py
-    function: analyze_data
-```
+### Mermaid Diagram: Story Generator Flow
 
 ```mermaid
-graph LR
-    A[Function Definition] --> B{Type: embedded/external};
-    B -- embedded --> C["Code (async def ...)"];
-    B -- external --> D[Module: PyPI, Path, or URL];
-    D --> E[Function Name];
-    style A fill:#ccf,stroke:#333,stroke-width:2px
+graph TD
+    A[generate_outline] --> B[generate_chapter]
+    B --> C[update_progress]
+    C --> D[check_if_complete]
+    D -->|"ctx['continue_generating']"| B
+    D -->|else| E[End]
+    style A fill:#e6ffe6,stroke:#009933,stroke-width:2px
+    style B fill:#e6ffe6,stroke:#009933,stroke-width:2px
+    style C fill:#e6ffe6,stroke:#009933,stroke-width:2px
+    style D fill:#e6ffe6,stroke:#009933,stroke-width:2px
+    style E fill:#fff0e6,stroke:#cc3300,stroke-width:2px
 ```
 
-## 4. Nodes
+#### Execution
+With `initial_context = {"genre": "science fiction", "num_chapters": 3, "chapters": [], "completed_chapters": 0, "style": "descriptive"}`:
+1. `generate_outline` creates an outline.
+2. `generate_chapter` writes a chapter.
+3. `update_progress` updates the chapter list and count.
+4. `check_if_complete` loops back if more chapters are needed.
 
-Nodes represent individual tasks within the workflow, configurable as function executions, sub-workflows, or LLM operations.
+---
 
-**Fields**
+## 4. Functions ⚙️
 
-*   `function` (string, optional): References a function from the `functions` section. Mutually exclusive with `sub_workflow` and `llm_config`.
-*   `sub_workflow` (object, optional): Defines a nested workflow. Mutually exclusive with `function` and `llm_config`.
-    *   `start` (string, required): Starting node of the sub-workflow.
-    *   `transitions` (list): Transition rules (see Workflow section).
-*   `llm_config` (object, optional): Configures an LLM-based node. Mutually exclusive with `function` and `sub_workflow`.
-    *   `model` (string, optional, default: `"gpt-3.5-turbo"`): LLM model (e.g., `"gemini/gemini-2.0-flash"`, `"gro k/xai"`).
-    *   `system_prompt` (string, optional): Defines the LLM’s role or context.
-    *   `prompt_template` (string, required, default: `"{{ input }}"`): Jinja2 template for the user prompt (e.g., `"Summarize {{ text }}"`).
-    *   `temperature` (float, optional, default: `0.7`): Randomness control (`0.0` to `1.0`).
-    *   `max_tokens` (integer, optional, default: `2000`): Maximum response tokens.
-    *   `top_p` (float, optional, default: `1.0`): Nucleus sampling (`0.0` to `1.0`).
-    *   `presence_penalty` (float, optional, default: `0.0`): Penalizes repetition (`-2.0` to `2.0`).
-    *   `frequency_penalty` (float, optional, default: `0.0`): Reduces word repetition (`-2.0` to `2.0`).
-    *   `stop` (list of strings, optional): Stop sequences (e.g., `["\n"]`).
-    *   `response_model` (string, optional): Pydantic model path for structured output (e.g., `"my_module:OrderDetails"`). If present, uses `structured_llm_node`; otherwise, uses `llm_node`.
-    *   `api_key` (string, optional): Custom API key for the LLM provider.
-*   `output` (string, optional): Context key for the node’s result. Defaults to `"<node_name>_result"` for function or LLM nodes if unspecified.
-*   `retries` (integer, optional, default: `3`): Number of retry attempts on failure (≥ `0`).
-*   `delay` (float, optional, default: `1.0`): Delay between retries in seconds (≥ `0`).
-*   `timeout` (float or null, optional, default: `null`): Execution timeout in seconds (≥ `0` or `null` for no timeout).
-*   `parallel` (boolean, optional, default: `false`): Enables parallel execution with other nodes.
+The `functions` section defines Python code for reuse.
 
-**Rules**
+### Fields 📋
+- `type` (string, required): `"embedded"` or `"external"`.
+- `code` (string, optional): Inline code for `embedded`.
+- `module` (string, optional): Source for `external` (PyPI, path, URL).
+- `function` (string, optional): Function name in `module`.
 
-*   Each node must specify exactly one of `function`, `sub_workflow`, or `llm_config`.
-*   For `sub_workflow`, `output` is optional if the sub-workflow sets multiple context keys; inputs are derived from the start node’s requirements.
-*   For `llm_config`, inputs are extracted from `prompt_template` placeholders (e.g., `{{ text }}` implies `text` as an input).
-
-**Examples**
-
-*   **Function Node**
+### Rules ✅
+- Embedded: Use `async def`, name matches key.
+- External: Requires `module` and `function`, no `code`.
 
+### Examples 🌈
+From the story generator:
 ```yaml
-nodes:
-  validate:
-    function: validate_order
-    output: is_valid
-    retries: 2
-    delay: 0.5
-    timeout: 5.0
+functions:
+  update_progress:
+    type: embedded
+    code: |
+      async def update_progress(**context):
+          chapters = context.get('chapters', [])
+          completed_chapters = context.get('completed_chapters', 0)
+          chapter = context.get('chapter', '')
+          return {**context, "chapters": chapters + [chapter], "completed_chapters": completed_chapters + 1}
 ```
-
-*   **Sub-Workflow Node**
-
+External example:
 ```yaml
-nodes:
-  payment_shipping:
-    sub_workflow:
-      start: payment
-      transitions:
-        - from: payment
-          to: shipping
-    output: shipping_confirmation
+functions:
+  fetch:
+    type: external
+    module: requests
+    function: get
 ```
 
-*   **Plain LLM Node**
-
-```yaml
-nodes:
-  summarize:
-    llm_config:
-      model: "gro k/xai"
-      system_prompt: "You are a concise summarizer."
-      prompt_template: "Summarize this text: {{ text }}"
-      temperature: 0.5
-      max_tokens: 50
-    output: summary
+```mermaid
+graph TD
+    A[Function Definition] --> B{Type?}
+    B -->|embedded| C[Code: async def ...]
+    B -->|external| D[Module: PyPI, Path, URL]
+    D --> E[Function Name]
+    style A fill:#e6f3ff,stroke:#0066cc,stroke-width:2px
+    style B fill:#fff,stroke:#333
+    style C fill:#cce6ff,stroke:#0066cc
+    style D fill:#cce6ff,stroke:#0066cc
+    style E fill:#cce6ff,stroke:#0066cc
 ```
 
-*   **Structured LLM Node**
-
+---
+
+## 5. Nodes 🧩
+
+Nodes are the tasks, powered by functions, sub-workflows, or LLMs.
+
+### Fields 📋
+- `function` (string, optional): Links to `functions`.
+- `sub_workflow` (object, optional):
+  - `start` (string)
+  - `transitions` (list)
+- `llm_config` (object, optional):
+  - `model` (string, default: `"gpt-3.5-turbo"`)
+  - `system_prompt` (string, optional)
+  - `prompt_template` (string, default: `"{{ input }}"`)
+  - `temperature` (float, default: `0.7`)
+  - `max_tokens` (int, optional)
+  - `top_p` (float, default: `1.0`)
+  - `presence_penalty` (float, default: `0.0`)
+  - `frequency_penalty` (float, default: `0.0`)
+  - `response_model` (string, optional)
+- `output` (string, optional): Context key.
+- `retries` (int, default: `3`)
+- `delay` (float, default: `1.0`)
+- `timeout` (float/null, default: `null`)
+- `parallel` (bool, default: `false`)
+
+### Rules ✅
+- One of `function`, `sub_workflow`, or `llm_config` per node.
+- LLM inputs come from `prompt_template`.
+
+### Examples 🌈
+From the story generator:
 ```yaml
 nodes:
-  check_inventory:
+  generate_outline:
     llm_config:
       model: "gemini/gemini-2.0-flash"
-      system_prompt: "Check inventory status."
-      prompt_template: "Are {{ items }} in stock?"
-      response_model: "my_module:InventoryStatus"
-    output: inventory_status
+      system_prompt: "You are a creative writer skilled at generating stories."
+      prompt_template: "Create a story outline for a {genre} story with {num_chapters} chapters."
+      temperature: 0.7
+      max_tokens: 1000
+    output: outline
 ```
 
 ```mermaid
-graph LR
-    A[Node Definition] --> B{Choice: function, sub_workflow, llm_config};
-    B -- function --> C[Function Name];
-    B -- sub_workflow --> D[Start Node & Transitions];
-    B -- llm_config --> E[LLM Configuration];
-    style A fill:#ccf,stroke:#333,stroke-width:2px
+graph TD
+    A[Node] --> B{Type?}
+    B -->|function| C[Function Ref]
+    B -->|sub_workflow| D[Start + Transitions]
+    B -->|llm_config| E[LLM Setup]
+    E --> F{Structured?}
+    F -->|Yes| G[response_model]
+    F -->|No| H[Plain Text]
+    style A fill:#e6ffe6,stroke:#009933,stroke-width:2px
+    style B fill:#fff,stroke:#333
+    style C fill:#ccffcc,stroke:#009933
+    style D fill:#ccffcc,stroke:#009933
+    style E fill:#ccffcc,stroke:#009933
+    style F fill:#fff,stroke:#333
+    style G fill:#b3ffb3,stroke:#009933
+    style H fill:#b3ffb3,stroke:#009933
 ```
 
-## 5. Workflow
-
-The `workflow` section defines the top-level execution flow.
-
-**Fields**
-
-*   `start` (string, optional): Name of the starting node.
-*   `transitions` (list, required): List of transition rules.
+---
 
-**Transition Fields**
+## 6. Workflow 🌐
 
-*   `from` (string, required): Source node.
-*   `to` (string or list, required): Target node(s). String for sequential, list for parallel execution.
-*   `condition` (string, optional): Python expression using `ctx` (e.g., `"ctx.get('in_stock')"`). Transition occurs if `True`.
+The `workflow` section defines execution order.
 
-**Examples**
-
-*   **Sequential Transition**
+### Fields 📋
+- `start` (string, optional): First node.
+- `transitions` (list):
+  - `from_node` (string)
+  - `to_node` (string/list)
+  - `condition` (string, optional)
 
+### Example 🌈
+From the story generator:
 ```yaml
 workflow:
-  start: validate
+  start: generate_outline
   transitions:
-    - from: validate
-      to: check_inventory
+    - from_node: generate_outline
+      to_node: generate_chapter
+    - from_node: generate_chapter
+      to_node: update_progress
+    - from_node: update_progress
+      to_node: check_if_complete
+    - from_node: check_if_complete
+      to_node: generate_chapter
+      condition: "ctx['continue_generating']"
 ```
 
-*   **Conditional Transition**
-
-```yaml
-workflow:
-  start: check_inventory
-  transitions:
-    - from: check_inventory
-      to: payment_shipping
-      condition: "ctx.get('inventory_status').in_stock"
+```mermaid
+graph TD
+    A[Workflow] --> B[Start Node]
+    A --> C[Transitions]
+    C --> D[From Node]
+    D --> E{To Node}
+    E -->|Sequential| F[Single Node]
+    E -->|Parallel| G[List of Nodes]
+    C --> H[Condition?]
+    H -->|Yes| I[ctx-based Logic]
+    style A fill:#fff0e6,stroke:#cc3300,stroke-width:2px
+    style B fill:#ffe6cc,stroke:#cc3300
+    style C fill:#ffe6cc,stroke:#cc3300
+    style D fill:#ffd9b3,stroke:#cc3300
+    style E fill:#fff,stroke:#333
+    style F fill:#ffd9b3,stroke:#cc3300
+    style G fill:#ffd9b3,stroke:#cc3300
+    style H fill:#fff,stroke:#333
+    style I fill:#ffd9b3,stroke:#cc3300
 ```
 
-*   **Parallel Transition**
+---
 
-```yaml
-workflow:
-  start: payment_shipping
-  transitions:
-    - from: payment_shipping
-      to: [update_status, notify_customer]
-```
+## 7. Workflow Validation 🕵️‍♀️
 
-```mermaid
-graph LR
-    A[Workflow Definition] --> B(Start Node);
-    A --> C(Transitions);
-    C --> D{From Node};
-    D --> E{To Nodes};
-    E -- Sequential --> F[Single Node];
-    E -- Parallel --> G[List of Nodes];
-    C --> H{Condition Optional};
-    style A fill:#ccf,stroke:#333,stroke-width:2px
+`validate_workflow_definition()` ensures integrity:
+- Checks node connectivity, circular references, undefined nodes, and missing start.
+- Returns `WorkflowIssue` objects (`node_name`, `description`).
+
+### Example
+```python
+issues = validate_workflow_definition(workflow)
+if issues:
+    for issue in issues:
+        print(f"Node '{issue.node_name}': {issue.description}")
 ```
 
-## 6. Context
+---
 
-The context (`ctx`) is a dictionary shared across the workflow and sub-workflows, storing node outputs. Examples:
+## 8. Observers 👀
 
-*   Function node: `ctx["is_valid"] = True`.
-*   Plain LLM node: `ctx["summary"] = "Brief text"`.
-*   Structured LLM node: `ctx["inventory_status"] = InventoryStatus(items=["item1"], in_stock=True)`.
+Monitor events like node starts or failures.
 
-## 7. Execution Flow
+### Example
+From the story generator:
+```yaml
+observers:
+  - story_observer
+```
 
-The `WorkflowEngine` executes the workflow as follows:
+---
 
-1.  Begins at `workflow.start`.
-2.  Executes nodes, updating `ctx`:
-    *   **Function Nodes**: Calls the referenced function, storing the result in `output`.
-    *   **Sub-Workflow Nodes**: Runs the nested workflow, merging its context into the parent’s.
-    *   **LLM Nodes**: Uses `Nodes.llm_node` for text output or `Nodes.structured_llm_node` for structured output via `litellm` and `instructor`.
-3.  Evaluates transitions:
-    *   Conditions (if present) are checked against `ctx`.
-4.  Executes the next node(s) sequentially or in parallel based on `to`.
-5.  Continues until no further transitions remain.
+## 9. Context 📦
 
-## 8. WorkflowManager
+The `ctx` dictionary shares data:
+- `generate_outline` → `ctx["outline"]`
+- `update_progress` → `ctx["chapters"]`, `ctx["completed_chapters"]`
 
-The `WorkflowManager` class provides programmatic control over workflows:
+---
 
-*   **Node Management**: Add, update, or remove nodes.
-*   **Transition Management**: Define execution flow.
-*   **Function Registration**: Embed or link to external functions.
-*   **YAML I/O**: Load/save workflows from/to YAML files.
-*   **Instantiation**: Builds a `Workflow` object with support for PyPI modules.
+## 10. Execution Flow 🏃‍♂️
 
-**Example**
+The `WorkflowEngine`:
+1. Starts at `workflow.start`.
+2. Executes nodes, updates `ctx`.
+3. Follows transitions based on conditions.
+4. Notifies observers.
+5. Ends when transitions are exhausted.
 
-```python
-manager = WorkflowManager()
-manager.add_function("fetch", "external", module="requests", function="get")
-manager.add_node("start", function="fetch", output="response")
-manager.set_start_node("start")
-manager.save_to_yaml("workflow.yaml")
-```
+---
 
-If `requests` is missing, the manager raises:
+## 11. Converting Between Python and YAML 🔄
 
-```text
-Failed to import module 'requests': No module named 'requests'. This may be a PyPI package. Ensure it is installed using 'pip install requests' or check if the module name is correct.
-```
+### Python to YAML (`flow_extractor.py`)
+```python
+from quantalogic.flow.flow_extractor import extract_workflow_from_file
+from quantalogic.flow.flow_manager import WorkflowManager
 
-```mermaid
-graph LR
-    A[WorkflowManager] --> B(Add/Update/Remove Nodes & Transitions);
-    A --> C(Load/Save YAML);
-    A --> D(Instantiate Workflow);
-    style A fill:#ccf,stroke:#333,stroke-width:2px
+wf_def, globals = extract_workflow_from_file("story_generator_agent.py")
+WorkflowManager(wf_def).save_to_yaml("story_generator_workflow.yaml")
 ```
 
-## 9. Examples
-
-**Example 1: Simple Workflow with PyPI Module**
+### YAML to Python (`flow_generator.py`)
+```python
+from quantalogic.flow.flow_generator import generate_executable_script
 
-```yaml
-functions:
-  fetch_page:
-    type: external
-    module: requests
-    function: get
-nodes:
-  fetch:
-    function: fetch_page
-    output: page_content
-workflow:
-  start: fetch
-  transitions: []
+manager = WorkflowManager().load_from_yaml("story_generator_workflow.yaml")
+generate_executable_script(manager.workflow, {}, "standalone_story.py")
 ```
 
-Execution with `ctx = {"url": "https://example.com"}`:
+```mermaid
+graph TD
+    A[Python Workflow] -->|flow_extractor.py| B[WorkflowDefinition]
+    B -->|WorkflowManager| C[YAML File]
+    C -->|WorkflowManager| D[WorkflowDefinition]
+    D -->|flow_generator.py| E[Standalone Python Script]
+    style A fill:#e6f3ff,stroke:#0066cc,stroke-width:2px
+    style B fill:#fff,stroke:#333
+    style C fill:#e6ffe6,stroke:#009933,stroke-width:2px
+    style D fill:#fff,stroke:#333
+    style E fill:#fff0e6,stroke:#cc3300,stroke-width:2px
+```
 
-`fetch` → `ctx["page_content"] = <Response object>` (assuming `requests` is installed).
+---
 
-**Example 2: E-commerce Workflow**
+## 12. WorkflowManager 🧑‍💻
 
-```yaml
-functions:
-  validate_order:
-    type: embedded
-    code: |
-      async def validate_order(order: dict) -> bool:
-          await asyncio.sleep(1)
-          return bool(order.get("items"))
-  process_payment:
-    type: external
-    module: stripe
-    function: create_charge
-nodes:
-  validate:
-    function: validate_order
-    output: is_valid
-  inventory:
-    llm_config:
-      model: "gemini/gemini-2.0-flash"
-      system_prompt: "Check inventory."
-      prompt_template: "Are {{ items }} in stock?"
-      response_model: "my_module:InventoryStatus"
-    output: inventory_status
-  payment:
-    function: process_payment
-    output: payment_status
-  notify:
-    llm_config:
-      prompt_template: "Notify: Order {{ order_id }} processed."
-    output: notification
-workflow:
-  start: validate
-  transitions:
-    - from: validate
-      to: inventory
-    - from: inventory
-      to: payment
-      condition: "ctx.get('inventory_status').in_stock"
-    - from: payment
-      to: notify
+Programmatic workflow creation:
+```python
+manager = WorkflowManager()
+manager.add_node("start", llm_config={"model": "grok/xai", "prompt_template": "Say hi"})
+manager.set_start_node("start")
+manager.save_to_yaml("hi.yaml")
 ```
 
-Execution with `ctx = {"order": {"items": ["item1"], "order_id": "123"}}`:
+---
 
-*   `validate` → `ctx["is_valid"] = True`.
-*   `inventory` → `ctx["inventory_status"] = InventoryStatus(...)`.
-*   `payment` → `ctx["payment_status"] = <Stripe response>` (requires `pip install stripe`).
-*   `notify` → `ctx["notification"] = "Notify: Order 123 processed."`.
-
-```mermaid
-graph LR
-    A[validate] --> B[inventory];
-    B -- "ctx.get('inventory_status').in_stock" --> C[payment];
-    C --> D[notify];
-    style A fill:#afa,stroke:#333,stroke-width:2px
-    style B fill:#afa,stroke:#333,stroke-width:2px
-    style C fill:#afa,stroke:#333,stroke-width:2px
-    style D fill:#afa,stroke:#333,stroke-width:2px
-```
+## 13. Conclusion 🎉
 
-## 10. Conclusion
+The Quantalogic Flow YAML DSL (March 2, 2025) is a powerful tool for workflow automation, exemplified by the Story Generator case study. With support for LLMs, flexible flows, and conversion tools, it bridges Python and YAML seamlessly. Whether you’re crafting stories or processing orders, this DSL, paired with `WorkflowManager`, is your key to efficient, scalable workflows. 🚀
 
-The Quantalogic Flow YAML DSL, as of February 23, 2025, provides a flexible and powerful framework for defining workflows. Enhanced support for PyPI modules via the `module` field in `functions` ensures seamless integration with external libraries, with clear error messages guiding users to install missing packages (e.g., `pip install requests`). Combined with sub-workflows, LLM nodes, and robust execution controls, it supports a wide range of applications, from simple tasks to complex, AI-driven processes, all manageable through the `WorkflowManager`.