flock-core 0.3.6__tar.gz → 0.3.8__tar.gz

{flock_core-0.3.6 → flock_core-0.3.8}/.gitignore

@@ -27,6 +27,7 @@ share/python-wheels/
 MANIFEST
 logs/
 metrics/
+.claude/
 
 
 # PyInstaller

{flock_core-0.3.6 → flock_core-0.3.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: flock-core
-Version: 0.3.6
+Version: 0.3.8
 Summary: Declarative LLM Orchestration at Scale
 Author-email: Andre Ratzenberger <andre.ratzenberger@whiteduck.de>
 License-File: LICENSE

flock_core-0.3.8/docs/architecture/components.md

@@ -0,0 +1,477 @@
+# Architecture Components
+
+This document provides detailed information about the specific components of the Flock framework and how they interact with each other.
+
+## Flock
+
+The `Flock` class is the main orchestrator for creating and executing agents. It manages the registration of agents and tools, sets up the global context, and runs the agent workflows.
+
+```mermaid
+classDiagram
+    class Flock {
+        +model: str
+        +local_debug: bool
+        +enable_logging: bool
+        +enable_telemetry: bool
+        +temporal_executor_config: TemporalExecutorConfig
+        +telemetry_config: TelemetryConfig
+        +logger: Logger
+        +add_agent(agent: FlockAgent)
+        +add_tool(tool_name: str, tool: Callable)
+        +run(start_agent: FlockAgent, input: dict)
+        +run_async(start_agent: FlockAgent, input: dict)
+        +save_to_file(file_path: str)
+        +load_from_file(file_path: str)
+    }
+```
+
+### Key Features
+
+- **Agent Management**: Add and manage multiple agents in a single system.
+- **Tool Registration**: Register tools that agents can use during execution.
+- **Context Management**: Set up and maintain the global execution context.
+- **Workflow Execution**: Run agent workflows either locally or via Temporal.
+- **Serialization**: Save and load Flock instances to/from files.
+
+### Important Methods
+
+- `add_agent(agent)`: Add a new agent to the Flock system.
+- `add_tool(tool_name, tool)`: Register a tool with the Flock system.
+- `run(start_agent, input, ...)`: Run an agent workflow synchronously.
+- `run_async(start_agent, input, ...)`: Run an agent workflow asynchronously.
+- `save_to_file(file_path)`: Save the Flock instance to a file.
+- `load_from_file(file_path)`: Load a Flock instance from a file.
+
+## FlockAgent
+
+The `FlockAgent` class is the core, declarative base class for all agents in the Flock framework. It defines the agent's inputs, outputs, tools, and lifecycle hooks.
+
+```mermaid
+classDiagram
+    class FlockAgent {
+        +name: str
+        +model: str
+        +description: str
+        +input: str
+        +output: str
+        +tools: list
+        +hand_off: HandOff
+        +evaluator: FlockEvaluator
+        +modules: list
+        +initialize(inputs: dict)
+        +evaluate(inputs: dict)
+        +terminate(inputs: dict, result: dict)
+        +on_error(error: Exception, inputs: dict)
+        +run(inputs: dict)
+        +run_async(inputs: dict)
+        +add_module(module: FlockModule)
+        +save_to_file(file_path: str)
+        +load_from_file(file_path: str)
+    }
+```
+
+### Key Features
+
+- **Declarative Definition**: Define agents by their inputs and outputs.
+- **Type Hints**: Support for type hints and descriptions in input/output definitions.
+- **Tool Integration**: Use tools to extend agent capabilities.
+- **Lifecycle Hooks**: Hooks for initialization, termination, evaluation, and error handling.
+- **Module Support**: Attach modules to extend agent functionality.
+- **Serialization**: Save and load agent instances to/from files.
+
+### Important Properties
+
+- `name`: Unique identifier for the agent.
+- `model`: The model to use (e.g., 'openai/gpt-4o').
+- `description`: A human-readable description of the agent.
+- `input`: A comma-separated list of input keys with optional type hints and descriptions.
+- `output`: A comma-separated list of output keys with optional type hints and descriptions.
+- `tools`: An optional list of callable tools that the agent can leverage during execution.
+- `hand_off`: Specifies the next agent in the workflow or a callable that determines the handoff.
+- `evaluator`: The evaluator to use for agent evaluation.
+- `modules`: FlockModules attached to this agent.
+
+### Lifecycle Hooks
+
+- `initialize(inputs)`: Called when the agent starts running.
+- `evaluate(inputs)`: Called to evaluate inputs and produce outputs.
+- `terminate(inputs, result)`: Called when the agent finishes running.
+- `on_error(error, inputs)`: Called when an error occurs during agent execution.
+
+## FlockEvaluator
+
+The `FlockEvaluator` class is the base class for all evaluators in Flock. An evaluator is responsible for taking inputs and producing outputs using some evaluation strategy.
+
+```mermaid
+classDiagram
+    class FlockEvaluator {
+        +name: str
+        +config: FlockEvaluatorConfig
+        +evaluate(agent: FlockAgent, inputs: dict, tools: list)
+    }
+    
+    class DeclarativeEvaluator {
+        +evaluate(agent: FlockAgent, inputs: dict, tools: list)
+    }
+    
+    class NaturalLanguageEvaluator {
+        +evaluate(agent: FlockAgent, inputs: dict, tools: list)
+    }
+    
+    FlockEvaluator <|-- DeclarativeEvaluator
+    FlockEvaluator <|-- NaturalLanguageEvaluator
+```
+
+### Key Features
+
+- **Evaluation Strategy**: Define how inputs are evaluated to produce outputs.
+- **Configuration**: Configure the evaluator's behavior.
+- **Abstract Interface**: Provide a common interface for different evaluation strategies.
+
+### Important Methods
+
+- `evaluate(agent, inputs, tools)`: Evaluate inputs to produce outputs.
+
+### Built-in Evaluators
+
+- **DeclarativeEvaluator**: The default evaluator that uses a declarative approach to evaluation.
+- **NaturalLanguageEvaluator**: An evaluator that uses natural language for evaluation.
+
+## FlockModule
+
+The `FlockModule` class is the base class for all Flock modules. Modules can hook into agent lifecycle events and modify or enhance agent behavior.
+
+```mermaid
+classDiagram
+    class FlockModule {
+        +name: str
+        +config: FlockModuleConfig
+        +initialize(agent: FlockAgent, inputs: dict)
+        +pre_evaluate(agent: FlockAgent, inputs: dict)
+        +post_evaluate(agent: FlockAgent, inputs: dict, result: dict)
+        +terminate(agent: FlockAgent, inputs: dict, result: dict)
+        +on_error(agent: FlockAgent, error: Exception, inputs: dict)
+    }
+    
+    class MemoryModule {
+        +initialize(agent: FlockAgent, inputs: dict)
+        +pre_evaluate(agent: FlockAgent, inputs: dict)
+        +post_evaluate(agent: FlockAgent, inputs: dict, result: dict)
+    }
+    
+    class MetricsModule {
+        +initialize(agent: FlockAgent, inputs: dict)
+        +pre_evaluate(agent: FlockAgent, inputs: dict)
+        +post_evaluate(agent: FlockAgent, inputs: dict, result: dict)
+        +terminate(agent: FlockAgent, inputs: dict, result: dict)
+    }
+    
+    class OutputModule {
+        +post_evaluate(agent: FlockAgent, inputs: dict, result: dict)
+    }
+    
+    FlockModule <|-- MemoryModule
+    FlockModule <|-- MetricsModule
+    FlockModule <|-- OutputModule
+```
+
+### Key Features
+
+- **Lifecycle Hooks**: Hook into agent lifecycle events.
+- **State Management**: Maintain state across agent executions.
+- **Configuration**: Configure the module's behavior.
+- **Pluggable**: Easily attach to and detach from agents.
+
+### Lifecycle Hooks
+
+- `initialize(agent, inputs)`: Called when the agent starts running.
+- `pre_evaluate(agent, inputs)`: Called before agent evaluation, can modify inputs.
+- `post_evaluate(agent, inputs, result)`: Called after agent evaluation, can modify results.
+- `terminate(agent, inputs, result)`: Called when the agent finishes running.
+- `on_error(agent, error, inputs)`: Called when an error occurs during agent execution.
+
+### Built-in Modules
+
+- **MemoryModule**: Provides persistent memory for agents.
+- **MetricsModule**: Collects metrics about agent execution.
+- **OutputModule**: Formats and stores agent outputs.
+
+## HandOff
+
+The `HandOff` class is used to specify the next agent in a workflow and the input data to pass to it.
+
+```mermaid
+classDiagram
+    class HandOff {
+        +next_agent: Union[str, FlockAgent, Callable]
+        +input: dict
+        +context: dict
+    }
+```
+
+### Key Properties
+
+- `next_agent`: The next agent to invoke (either a string name, a FlockAgent instance, or a callable that returns an agent).
+- `input`: Input data for the next agent.
+- `context`: Override context parameters.
+
+## Registry
+
+The registry manages the registration of agents and tools.
+
+```mermaid
+classDiagram
+    class Registry {
+        +agents: dict
+        +tools: dict
+        +register_agent(agent: FlockAgent)
+        +register_tool(tool_name: str, tool: Callable)
+        +get_agent(agent_name: str)
+        +get_tool(tool_name: str)
+        +get_all_agents()
+        +get_all_tools()
+    }
+```
+
+### Key Features
+
+- **Agent Registration**: Register agents with the registry.
+- **Tool Registration**: Register tools with the registry.
+- **Lookup**: Look up agents and tools by name.
+
+### Important Methods
+
+- `register_agent(agent)`: Register an agent with the registry.
+- `register_tool(tool_name, tool)`: Register a tool with the registry.
+- `get_agent(agent_name)`: Get an agent by name.
+- `get_tool(tool_name)`: Get a tool by name.
+- `get_all_agents()`: Get all registered agents.
+- `get_all_tools()`: Get all registered tools.
+
+## Context
+
+The context maintains the global execution state.
+
+```mermaid
+classDiagram
+    class Context {
+        +variables: dict
+        +get(key: str, default=None)
+        +set(key: str, value)
+        +update(variables: dict)
+        +clear()
+    }
+```
+
+### Key Features
+
+- **Variable Storage**: Store global variables that are accessible to all agents.
+- **Variable Access**: Get and set context variables.
+- **Variable Lifecycle**: Manage the lifecycle of context variables.
+
+### Important Methods
+
+- `get(key, default=None)`: Get a context variable.
+- `set(key, value)`: Set a context variable.
+- `update(variables)`: Update multiple context variables.
+- `clear()`: Clear all context variables.
+
+## Executor
+
+The executor runs agent workflows. Flock supports two types of executors:
+
+```mermaid
+classDiagram
+    class Executor {
+        +execute(start_agent: FlockAgent, input: dict)
+        +execute_async(start_agent: FlockAgent, input: dict)
+    }
+    
+    class LocalExecutor {
+        +execute(start_agent: FlockAgent, input: dict)
+        +execute_async(start_agent: FlockAgent, input: dict)
+    }
+    
+    class TemporalExecutor {
+        +config: TemporalExecutorConfig
+        +execute(start_agent: FlockAgent, input: dict)
+        +execute_async(start_agent: FlockAgent, input: dict)
+    }
+    
+    Executor <|-- LocalExecutor
+    Executor <|-- TemporalExecutor
+```
+
+### Key Features
+
+- **Workflow Execution**: Execute agent workflows.
+- **Synchronous and Asynchronous Execution**: Support for both synchronous and asynchronous execution.
+- **Error Handling**: Handle errors during workflow execution.
+
+### Important Methods
+
+- `execute(start_agent, input)`: Execute a workflow synchronously.
+- `execute_async(start_agent, input)`: Execute a workflow asynchronously.
+
+### Executor Types
+
+- **LocalExecutor**: Runs workflows locally for development and testing.
+- **TemporalExecutor**: Runs workflows via Temporal for production-grade execution.
+
+## Router
+
+The router determines the next agent in a workflow.
+
+```mermaid
+classDiagram
+    class FlockRouter {
+        +name: str
+        +config: FlockRouterConfig
+        +route(agent: FlockAgent, result: dict)
+    }
+    
+    class DefaultRouter {
+        +route(agent: FlockAgent, result: dict)
+    }
+    
+    class LLMRouter {
+        +route(agent: FlockAgent, result: dict)
+    }
+    
+    class AgentRouter {
+        +route(agent: FlockAgent, result: dict)
+    }
+    
+    FlockRouter <|-- DefaultRouter
+    FlockRouter <|-- LLMRouter
+    FlockRouter <|-- AgentRouter
+```
+
+### Key Features
+
+- **Routing Strategy**: Define how to determine the next agent in a workflow.
+- **Configuration**: Configure the router's behavior.
+- **Abstract Interface**: Provide a common interface for different routing strategies.
+
+### Important Methods
+
+- `route(agent, result)`: Determine the next agent in the workflow.
+
+### Router Types
+
+- **DefaultRouter**: Uses the agent's `hand_off` property to determine the next agent.
+- **LLMRouter**: Uses an LLM to determine the next agent.
+- **AgentRouter**: Uses a dedicated agent to make routing decisions.
+
+## Tools
+
+Tools are functions that agents can use to perform specific tasks.
+
+```mermaid
+classDiagram
+    class Tool {
+        +name: str
+        +description: str
+        +function: Callable
+        +parameters: dict
+        +returns: dict
+    }
+```
+
+### Key Features
+
+- **Function Wrapping**: Wrap Python functions as tools.
+- **Documentation**: Provide documentation for tools.
+- **Parameter Validation**: Validate tool parameters.
+- **Result Formatting**: Format tool results.
+
+### Built-in Tools
+
+- **Web Search**: Search the web for information.
+- **Code Evaluation**: Evaluate Python code.
+- **Web Content Retrieval**: Get content from web pages.
+- **Math Evaluation**: Evaluate mathematical expressions.
+- **Utility Functions**: Various utility functions like getting the current time.
+
+## API Server
+
+The API server allows you to expose your agents via HTTP endpoints.
+
+```mermaid
+classDiagram
+    class FlockAPI {
+        +flock: Flock
+        +app: FastAPI
+        +start(host: str, port: int)
+        +run_agent(agent_name: str, inputs: dict, async_run: bool)
+        +run_flock(agent_name: str, inputs: dict, async_run: bool)
+        +get_run_status(run_id: str)
+        +list_agents()
+    }
+```
+
+### Key Features
+
+- **HTTP Endpoints**: Expose agents via HTTP endpoints.
+- **Synchronous and Asynchronous Execution**: Support for both synchronous and asynchronous execution.
+- **Run Status Tracking**: Track the status of runs.
+- **Agent Discovery**: List available agents.
+
+### Important Methods
+
+- `start(host, port)`: Start the API server.
+- `run_agent(agent_name, inputs, async_run)`: Run an agent.
+- `run_flock(agent_name, inputs, async_run)`: Run a Flock workflow.
+- `get_run_status(run_id)`: Get the status of a run.
+- `list_agents()`: List available agents.
+
+## Component Interactions
+
+The following diagram shows how the components interact with each other during a typical workflow execution:
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant Flock
+    participant Registry
+    participant Context
+    participant Executor
+    participant Router
+    participant Agent1
+    participant Evaluator1
+    participant Module1
+    participant Agent2
+    participant Evaluator2
+    participant Module2
+    
+    User->>Flock: Create Flock instance
+    User->>Flock: Add agents
+    Flock->>Registry: Register agents
+    User->>Flock: Run workflow
+    Flock->>Context: Initialize context
+    Flock->>Executor: Execute workflow
+    Executor->>Agent1: Execute agent
+    Agent1->>Module1: Initialize
+    Agent1->>Evaluator1: Evaluate
+    Evaluator1->>Agent1: Return result
+    Agent1->>Module1: Post-evaluate
+    Agent1->>Router: Determine next agent
+    Router->>Agent1: Return next agent
+    Agent1->>Agent2: Hand off
+    Agent2->>Module2: Initialize
+    Agent2->>Evaluator2: Evaluate
+    Evaluator2->>Agent2: Return result
+    Agent2->>Module2: Post-evaluate
+    Agent2->>Executor: Return result
+    Executor->>Flock: Return result
+    Flock->>User: Return result
+```
+
+## Next Steps
+
+Now that you understand the specific components of Flock, you might want to explore:
+
+- [Overview](overview.md) - Get a high-level overview of the Flock architecture
+- [Design Decisions](design-decisions.md) - Understand the design decisions behind Flock
+- [Core Concepts](../core-concepts/agents.md) - Learn about the core concepts of Flock
+- [Examples](../examples/hello-flock.md) - See examples of Flock in action

flock_core-0.3.8/docs/architecture/overview.md

@@ -0,0 +1,216 @@
+# Architecture Overview
+
+This document provides a high-level overview of the Flock framework's architecture, explaining the key components and how they interact with each other.
+
+```mermaid
+flowchart TD
+    A[Flock] --> B[FlockAgent]
+    A --> C[Registry]
+    A --> D[Context]
+    A --> E[Executor]
+    
+    B --> F[Evaluator]
+    B --> G[Modules]
+    B --> H[Tools]
+    B --> I[HandOff]
+    
+    E --> J[Local Executor]
+    E --> K[Temporal Executor]
+    
+    F --> L[Declarative Evaluator]
+    F --> M[Natural Language Evaluator]
+    F --> N[Custom Evaluators]
+    
+    G --> O[Memory Module]
+    G --> P[Metrics Module]
+    G --> Q[Output Module]
+    G --> R[Custom Modules]
+    
+    C --> S[Agent Registry]
+    C --> T[Tool Registry]
+    
+    style A fill:#f9d77e,stroke:#333,stroke-width:2px
+    style B,C,D,E fill:#7ecef9,stroke:#333,stroke-width:2px
+    style F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T fill:#7ef9a2,stroke:#333,stroke-width:2px
+```
+
+## Core Architecture
+
+Flock is designed with a modular architecture that separates concerns and allows for flexibility and extensibility. The main components are:
+
+1. **Flock**: The main orchestrator that manages agents, tools, and workflows.
+2. **FlockAgent**: The base class for all agents, defining their inputs, outputs, and behavior.
+3. **Registry**: Manages the registration of agents and tools.
+4. **Context**: Maintains the global execution context.
+5. **Executor**: Runs agent workflows either locally or via Temporal.
+
+## Component Relationships
+
+### Flock
+
+The `Flock` class is the main entry point for the framework. It:
+
+- Manages a collection of agents
+- Registers tools that agents can use
+- Maintains the global context
+- Orchestrates the execution of workflows
+
+### FlockAgent
+
+The `FlockAgent` class is the core, declarative base class for all agents. It:
+
+- Defines the agent's inputs and outputs
+- Specifies the tools the agent can use
+- Defines the agent's behavior through lifecycle hooks
+- Manages the agent's modules
+- Specifies the next agent in the workflow (handoff)
+
+### Registry
+
+The registry manages the registration of agents and tools. It:
+
+- Maintains a mapping of agent names to agent instances
+- Maintains a mapping of tool names to tool functions
+- Provides methods for registering and retrieving agents and tools
+
+### Context
+
+The context maintains the global execution state. It:
+
+- Stores global variables that are accessible to all agents
+- Provides methods for getting and setting context variables
+- Manages the lifecycle of context variables
+
+### Executor
+
+The executor runs agent workflows. Flock supports two types of executors:
+
+- **Local Executor**: Runs workflows locally for development and testing
+- **Temporal Executor**: Runs workflows via Temporal for production-grade execution
+
+## Execution Flow
+
+The execution flow in Flock follows these steps:
+
+1. **Initialization**: The Flock instance is created and agents are added.
+2. **Registration**: Agents and tools are registered with the registry.
+3. **Context Setup**: The global context is initialized with standardized variables.
+4. **Workflow Execution**: The agent workflow is executed either locally or via Temporal.
+5. **Agent Execution**: Each agent in the workflow is executed in sequence.
+6. **Handoff**: Agents can hand off to other agents to continue the workflow.
+7. **Termination**: The workflow terminates when the last agent completes or an error occurs.
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant Flock
+    participant Registry
+    participant Context
+    participant Executor
+    participant Agent1
+    participant Agent2
+    
+    User->>Flock: Create Flock instance
+    User->>Flock: Add agents
+    Flock->>Registry: Register agents
+    User->>Flock: Run workflow
+    Flock->>Context: Initialize context
+    Flock->>Executor: Execute workflow
+    Executor->>Agent1: Execute agent
+    Agent1->>Agent1: Initialize
+    Agent1->>Agent1: Evaluate
+    Agent1->>Agent1: Terminate
+    Agent1->>Agent2: Hand off
+    Agent2->>Agent2: Initialize
+    Agent2->>Agent2: Evaluate
+    Agent2->>Agent2: Terminate
+    Agent2->>Executor: Return result
+    Executor->>Flock: Return result
+    Flock->>User: Return result
+```
+
+## Evaluators
+
+Evaluators are responsible for evaluating agent inputs to produce outputs. Flock supports different types of evaluators:
+
+- **Declarative Evaluator**: The default evaluator that uses a declarative approach to evaluation.
+- **Natural Language Evaluator**: An evaluator that uses natural language for evaluation.
+- **Custom Evaluators**: You can create custom evaluators by subclassing `FlockEvaluator`.
+
+## Modules
+
+Modules are pluggable components that can hook into agent lifecycle events. Flock includes several built-in modules:
+
+- **Memory Module**: Provides persistent memory for agents.
+- **Metrics Module**: Collects metrics about agent execution.
+- **Output Module**: Formats and stores agent outputs.
+- **Custom Modules**: You can create custom modules by subclassing `FlockModule`.
+
+## Tools
+
+Tools are functions that agents can use to perform specific tasks. Flock includes several built-in tools:
+
+- **Web Search**: Search the web for information.
+- **Code Evaluation**: Evaluate Python code.
+- **Web Content Retrieval**: Get content from web pages.
+- **Math Evaluation**: Evaluate mathematical expressions.
+- **Utility Functions**: Various utility functions like getting the current time.
+- **Custom Tools**: You can create custom tools by defining Python functions.
+
+## Routers
+
+Routers determine the next agent in a workflow. Flock supports different types of routers:
+
+- **Default Router**: Uses the agent's `hand_off` property to determine the next agent.
+- **LLM Router**: Uses an LLM to determine the next agent.
+- **Agent Router**: Uses a dedicated agent to make routing decisions.
+- **Custom Routers**: You can create custom routers by subclassing `FlockRouter`.
+
+## Temporal Integration
+
+Flock integrates with Temporal for production-grade workflow execution. This provides:
+
+- **Durability**: Persistent state management even in the case of failures.
+- **Retries & Error Handling**: Automatic recovery via Temporal's built-in mechanisms.
+- **Scalability**: Seamless orchestration of distributed agent workflows.
+
+## API Server
+
+Flock includes a REST API server that allows you to expose your agents via HTTP endpoints. This makes it easy to integrate Flock agents into web applications, microservices, or any system that can make HTTP requests.
+
+## Design Principles
+
+Flock is designed with the following principles in mind:
+
+### Declarative Approach
+
+Flock takes a declarative approach to agent definition, focusing on what agents need as input and what they produce as output, rather than requiring complex prompt engineering.
+
+### Type Safety
+
+Flock uses Pydantic models for type safety and clear contracts for inputs and outputs. This provides automatic JSON serialization/deserialization, strong typing, and an explicit contract for inputs and outputs.
+
+### Lifecycle Hooks
+
+Flock agents support lifecycle hooks such as `initialize()`, `terminate()`, `evaluate()`, and `on_error()`. This ensures that agents can perform setup, cleanup, and robust error handling—all without cluttering the main business logic.
+
+### Fault Tolerance
+
+Flock integrates with Temporal for fault tolerance and production readiness. This provides automatic retries, durable state management, and resilience against failures.
+
+### Modularity
+
+Flock is designed to be modular, with clear separation of concerns. This makes it easy to extend and customize the framework for specific use cases.
+
+### Flexibility
+
+Flock is designed to be flexible, allowing for a wide range of agent types and workflows. Agents can be chained together, run in parallel, or run in a batch mode.
+
+## Next Steps
+
+Now that you understand the high-level architecture of Flock, you might want to explore:
+
+- [Components](components.md) - Learn more about the specific components of Flock
+- [Design Decisions](design-decisions.md) - Understand the design decisions behind Flock
+- [Core Concepts](../core-concepts/agents.md) - Learn about the core concepts of Flock
+- [Examples](../examples/hello-flock.md) - See examples of Flock in action