flock-core 0.2.1__tar.gz → 0.2.2__tar.gz

{flock_core-0.2.1 → flock_core-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: flock-core
-Version: 0.2.1
+Version: 0.2.2
 Summary: Declarative LLM Orchestration at Scale
 Author-email: Andre Ratzenberger <andre.ratzenberger@whiteduck.de>
 License-File: LICENSE
@@ -14,6 +14,11 @@ Requires-Dist: dspy==2.5.42
 Requires-Dist: httpx>=0.28.1
 Requires-Dist: loguru>=0.7.3
 Requires-Dist: msgpack>=1.1.0
+Requires-Dist: opentelemetry-api>=1.30.0
+Requires-Dist: opentelemetry-exporter-jaeger>=1.21.0
+Requires-Dist: opentelemetry-exporter-otlp>=1.30.0
+Requires-Dist: opentelemetry-instrumentation-logging>=0.51b0
+Requires-Dist: opentelemetry-sdk>=1.30.0
 Requires-Dist: pydantic>=2.10.5
 Requires-Dist: python-box>=7.3.2
 Requires-Dist: rich>=13.9.4
@@ -30,7 +35,7 @@ Description-Content-Type: text/markdown
 
 <p align="center">
 <img src="docs/img/flock.png" width="600"><br>
-<h1 align="center">Flock</h1><br>
+<h1 align="center">Flock<br></h1><br>
 
 
 ## Overview
@@ -83,6 +88,10 @@ Flock is a framework for orchestrating LLM-powered agents. It leverages a **decl
 - **DSPy Integration:**  
   Flock leverages DSPy for managing LLM interactions. The framework constructs clean signature strings and updates field metadata so that DSPy can include detailed instructions and context for each agent call.
 
+
+<p align="center">
+<img src="docs/img/flock_cli.png" width="200"><br>
+
 ## Quick Start
 
 Below is a simple example of how to create and run an agent with Flock:
@@ -238,6 +247,30 @@ pip install flock-core[all-tools]
    uv build && uv pip install -e .
    ```
 
+Install Jaeger for telemetry
+```
+
+docker run -d --name jaeger \
+  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
+  -p 5775:5775/udp \
+  -p 6831:6831/udp \
+  -p 6832:6832/udp \
+  -p 5778:5778 \
+  -p 16686:16686 \
+  -p 14268:14268 \
+  -p 14250:14250 \
+  -p 9411:9411 \
+  jaegertracing/all-in-one:1.41
+
+
+```
+
+or zipkin
+
+```
+docker run -d -p 9411:9411 openzipkin/zipkin
+
+```
 
 ## Contributing
 

{flock_core-0.2.1 → flock_core-0.2.2}/README.md

@@ -1,6 +1,6 @@
 <p align="center">
 <img src="docs/img/flock.png" width="600"><br>
-<h1 align="center">Flock</h1><br>
+<h1 align="center">Flock<br></h1><br>
 
 
 ## Overview
@@ -53,6 +53,10 @@ Flock is a framework for orchestrating LLM-powered agents. It leverages a **decl
 - **DSPy Integration:**  
   Flock leverages DSPy for managing LLM interactions. The framework constructs clean signature strings and updates field metadata so that DSPy can include detailed instructions and context for each agent call.
 
+
+<p align="center">
+<img src="docs/img/flock_cli.png" width="200"><br>
+
 ## Quick Start
 
 Below is a simple example of how to create and run an agent with Flock:
@@ -208,6 +212,30 @@ pip install flock-core[all-tools]
    uv build && uv pip install -e .
    ```
 
+Install Jaeger for telemetry
+```
+
+docker run -d --name jaeger \
+  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
+  -p 5775:5775/udp \
+  -p 6831:6831/udp \
+  -p 6832:6832/udp \
+  -p 5778:5778 \
+  -p 16686:16686 \
+  -p 14268:14268 \
+  -p 14250:14250 \
+  -p 9411:9411 \
+  jaegertracing/all-in-one:1.41
+
+
+```
+
+or zipkin
+
+```
+docker run -d -p 9411:9411 openzipkin/zipkin
+
+```
 
 ## Contributing
 

flock_core-0.2.2/docs/agent_mechanics.md

@@ -0,0 +1,48 @@
+# The FlockAgent: Embracing AI's Nature
+
+You've seen them all - those LLM frameworks trying to tame the beast. 300-line prompts, complex state machines, and endless guardrails. But here's the thing: LLMs are fuzzy by nature. Their responses are "statistically likely but non-deterministic" - a developer's nightmare. Yet we keep trying to force them into our deterministic world, don't we?
+
+That's where Flock comes in with a radically different philosophy: Instead of fighting the fuzziness, let's work with it.
+
+Think about the current landscape. Graph-based frameworks that turn your "AI agents" into glorified state machines. So much guardrailing that by the time you're done, there's barely any AI left. You wanted agent-based architecture but what is left after all the hard work is something that basically amounts to a classic static service. Ironic, isn't it?
+
+Here's our take: Let AI be AI. But give it a proper framework to thrive in.
+
+You're the Q to your AI agents. You don't control James Bond's every move - you equip him with the right tools for the mission. That's exactly how Flock works. We're talking full-spectrum support here: From the agent-level tools that make your AI shine, all the way to battle-tested platform capabilities that keep your system running when things get rough. 
+
+No stones in your path, just pure flexibility. 
+
+**Why Callables Change Everything**
+
+Every aspect of a FlockAgent - inputs, outputs, lifecycle hooks - can be a callable. This isn't just some feature checkbox. It means:
+- Your agent adapts to context on the fly
+- Edge cases? Handle them at runtime
+- Complex logic? No problem - go beyond static configs
+
+**The Lifecycle Revolution**
+
+We give you hooks where they matter:
+- `initialize`: Set the stage (resources, validation, whatever you need)
+- `evaluate`: Where the magic happens
+- `terminate`: Clean up your act
+- `on_error`: Because stuff happens
+
+Now, I know what you're thinking: "This sounds complex." But here's the twist - this complexity is your friend. Clear contracts beat fuzzy prompts any day. Want to switch input parameters? One hook, clearly visible. Want to handle an edge case? Good luck finding it in that 1,500-word prompt you wrote last month.
+
+**The Beautiful Basics**
+
+At its core, a FlockAgent is beautifully simple:
+- Give it a name
+- Tell it what it needs (input)
+- Tell it what it should produce (output)
+
+That's it. The rest is up to you.
+
+The result? A framework that's actually ready for production. Your agents are:
+- Modular enough to chain like LEGO bricks
+- Reusable thanks to Pydantic's typing magic
+- Customizable down to their core
+
+You get to build real, scalable AI systems without the prompt engineering headaches. No more fighting the AI - just harnessing its power in a way that actually makes sense for production.
+
+This isn't just another agent framework. It's a fundamental rethinking of how we work with LLMs in production environments. Because sometimes, the best way to control chaos is to embrace it - with the right tools in hand.

{flock_core-0.2.1 → flock_core-0.2.2}/examples/01_introduction/01_simple_example.py

@@ -39,7 +39,7 @@ async def main():
     # The flock is the place where all the agents are at home
     # set local_debug to True to run the flock without Temporal
     # Check out the examples in /temporal to learn about Temporal
-    flock = Flock(model=MODEL, local_debug=True)
+    flock = Flock(model=MODEL, local_debug=True, enable_logging=True)
 
     # --------------------------------
     # Create an agent

{flock_core-0.2.1 → flock_core-0.2.2}/examples/02_cook_book_wip/project_manager.py

@@ -29,6 +29,7 @@ Let's see how it all comes together!
 
 
 import asyncio
+from dataclasses import dataclass
 
 
 from flock.core.flock import Flock
@@ -39,14 +40,21 @@ from flock.core.tools import basic_tools
 from flock.core.tools.dev_tools import github
 
 
+@dataclass
+class Features:
+    title: str
+    description: str
+    acceptance_criteria: str
+
+
 async def main():
 
-    flock = Flock(local_debug=True, output_formatter=FormatterOptions(formatter=RichTables, wait_for_input=True, settings={}),enable_logging=True)
+    flock = Flock(local_debug=True, output_formatter=FormatterOptions(formatter=RichTables, wait_for_input=False, settings={}),enable_logging=True)
     
     idea_agent = FlockAgent(
         name="idea_agent",
         input="query",
-        output="a_fun_software_project_idea",
+        output="software_project_idea",
         tools=[basic_tools.web_search_tavily],
         use_cache=True,
     )
@@ -67,9 +75,17 @@ async def main():
         use_cache=True,
     )   
 
+    feature_agent = FlockAgent(
+        name="feature_agent",
+        input="readme, catchy_project_name, project_pitch, techstack, project_implementation_plan",
+        output="features : list[Features]",
+        tools=[github.create_user_stories_as_github_issue, github.create_files],
+        use_cache=True,
+    )   
+
     issue_agent = FlockAgent(
         name="issue_agent",
-        input="readme, catchy_project_name, project_pitch, techstack, project_implementation_plan",
+        input="current_feature, readme, techstack, project_implementation_plan, all_feature_titles",
         output="user_stories_on_github, files_on_github",
         tools=[github.create_user_stories_as_github_issue, github.create_files],
         use_cache=True,
@@ -77,15 +93,16 @@ async def main():
 
     idea_agent.hand_off = project_plan_agent
     project_plan_agent.hand_off = readme_agent
-    readme_agent.hand_off = issue_agent
+    readme_agent.hand_off = feature_agent
+
 
     flock.add_agent(idea_agent)
     flock.add_agent(project_plan_agent)
     flock.add_agent(readme_agent)
+    flock.add_agent(feature_agent)
     flock.add_agent(issue_agent)
 
-    await flock.run_async(
-        input={"query": "fun software project idea"},
+    features : Features = await flock.run_async(
         start_agent=idea_agent,
     )
 

flock_core-0.2.2/examples/02_cook_book_wip/self_improvement.py

@@ -0,0 +1,104 @@
+"""
+Title: Reasoning assistant with self managed memory
+"""
+import warnings
+warnings.simplefilter("error", UserWarning)
+import asyncio
+from dataclasses import dataclass, field
+
+from flock.core.flock import Flock
+from flock.core.flock_agent import FlockAgent, FlockAgentConfig, HandOff
+
+from rich.prompt import Prompt
+from rich.panel import Panel
+from rich.console import Console
+
+
+@dataclass
+class Chat:
+    chat_history: list[str] = field(default_factory=list)
+    user_query: str = ""
+    answer_to_query: str = ""
+    memory: str = ""
+    
+    async def before_response(self, agent, inputs):
+        console = Console()
+        # Load the memory from file (if it exists)
+        try:
+            with open("memory.txt", "r") as file:
+                self.memory = file.read()
+        except FileNotFoundError:
+            self.memory = ""
+
+        # Use a Rich-styled prompt to get user input
+        self.user_query = Prompt.ask("[bold cyan]User[/bold cyan]")
+        inputs["user_query"] = self.user_query
+        inputs["chat_history"] = self.chat_history
+        inputs["memory"] = self.memory
+
+    # Triggers after the agent responds to the user query
+    async def after_response(self, agent, inputs, outputs):
+        # Update answer and history based on the agent's outputs
+        console = Console()
+        self.answer_to_query = outputs["answer_to_query"]
+        self.chat_history.append({"user": self.user_query, "assistant": self.answer_to_query})
+        self.memory += outputs.get("important_new_knowledge_to_add_to_memory", "") + "\n"
+
+        # Save updated memory to file
+        with open("memory.txt", "w") as file:
+            file.write(self.memory)
+
+        # Display the assistant's reasoning (if available) in a styled panel
+        reasoning = outputs.get("reasoning", "")
+        if reasoning:
+            reasoning_panel = Panel(
+                reasoning,
+                title="[bold blue]Assistant Reasoning[/bold blue]",
+                border_style="blue",
+            )
+            console.print(reasoning_panel)
+
+        # Display the assistant's answer in a styled panel
+        answer_panel = Panel(
+            self.answer_to_query,
+            title="[bold green]Assistant Answer[/bold green]",
+            border_style="green",
+        )
+        console.print(answer_panel)
+
+    # Triggers at handoff to the next agent
+    def hand_off(self, context, result):
+        if self.user_query.lower() == "goodbye":
+            return None
+        return HandOff(next_agent="chatty")
+
+
+MODEL = "openai/gpt-4o"
+
+async def main():
+
+    chat_helper = Chat()
+    flock = Flock(model=MODEL, local_debug=True)
+
+
+    chatty = FlockAgent(
+        name="chatty", 
+        description="You are Chatty, a friendly assistant that loves to chat.",
+        input="user_query, memory | Memory of previous interactions, chat_history | the current chat history", 
+        output="answer_to_query, important_new_knowledge_to_add_to_memory | Empty string if no knowledge to add",
+        initialize_callback=chat_helper.before_response,
+        terminate_callback=chat_helper.after_response,
+        config=FlockAgentConfig(disable_output=True,agent_type_override="ChainOfThought")
+    )
+    flock.add_agent(chatty)
+
+    chatty.hand_off = chat_helper.hand_off
+   
+    await flock.run_async(
+        start_agent=chatty, 
+        input={"memory": "","user_query": "","chat_history": ""}
+    )
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

{flock_core-0.2.1 → flock_core-0.2.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "flock-core"
-version = "0.2.1"
+version = "0.2.2"
 description = "Declarative LLM Orchestration at Scale"
 readme = "README.md"
 authors = [
@@ -19,6 +19,11 @@ dependencies = [
     "httpx>=0.28.1",
     "loguru>=0.7.3",
     "msgpack>=1.1.0",
+    "opentelemetry-api>=1.30.0",
+    "opentelemetry-exporter-jaeger>=1.21.0",
+    "opentelemetry-exporter-otlp>=1.30.0",
+    "opentelemetry-instrumentation-logging>=0.51b0",
+    "opentelemetry-sdk>=1.30.0",
     "pydantic>=2.10.5",
     "python-box>=7.3.2",
     "rich>=13.9.4",

flock_core-0.2.2/review/analysis.md

@@ -0,0 +1,134 @@
+# Flock Framework Analysis
+
+## Overview
+
+Flock is a sophisticated Python framework for building and orchestrating LLM-powered agents with a unique declarative approach. The framework emphasizes simplicity and clarity by moving away from traditional prompt engineering in favor of a more structured, type-driven architecture.
+
+## Core Architecture
+
+### 1. Flock Orchestrator (`flock.py`)
+
+The `Flock` class serves as the central orchestrator with several key responsibilities:
+
+- Agent and tool registration management
+- Context initialization and management
+- Workflow execution (both local and distributed via Temporal)
+- Configuration of logging and output formatting
+
+Key features:
+- Support for both local debugging and production (Temporal) execution modes
+- Integrated telemetry and tracing
+- Flexible output formatting
+- Dynamic agent registration and tool management
+
+### 2. Agent System (`flock_agent.py`)
+
+The `FlockAgent` class is the cornerstone of the framework, implementing a declarative pattern for agent definition. Notable aspects include:
+
+#### Declarative Design
+Instead of manual prompt engineering, agents are defined by declaring:
+- Input specifications (with types and descriptions)
+- Output specifications (with types and descriptions)
+- Optional tools for extended functionality
+
+#### Agent Lifecycle
+The framework implements a comprehensive lifecycle management system:
+1. Initialization phase (`initialize`)
+2. Evaluation phase (`evaluate`)
+3. Termination phase (`terminate`)
+4. Error handling (`on_error`)
+
+#### Serialization Support
+Built-in capabilities for agent serialization and deserialization:
+- `to_dict()`: Converts agent instances to dictionaries
+- `from_dict()`: Reconstructs agents from dictionary representations
+- Support for cloudpickle serialization of callable objects
+
+## Key Features
+
+### 1. Declarative Programming Model
+- Agents are defined by their input/output specifications rather than prompt templates
+- Type hints and descriptions are used to generate appropriate prompts automatically
+- Reduces boilerplate and improves maintainability
+
+### 2. Tool Integration
+- Flexible tool registration system
+- Support for both synchronous and asynchronous tools
+- Tools can be shared across agents
+
+### 3. Workflow Management
+- Support for agent chaining through handoffs
+- Built-in context management
+- Integrated with Temporal for production-grade workflow orchestration
+
+### 4. Development Experience
+- Local debugging mode for rapid development
+- Rich logging and telemetry integration
+- Comprehensive error handling
+
+## Technical Implementation
+
+### Context Management
+- Uses a `FlockContext` system for managing state
+- Supports both global and agent-specific contexts
+- Implements context initialization and validation
+
+### Execution Models
+1. Local Execution (`run_local_workflow`)
+   - Simplified execution for development and debugging
+   - Direct agent interaction
+
+2. Temporal Execution (`run_temporal_workflow`)
+   - Production-grade workflow execution
+   - Fault tolerance and scalability
+   - Distributed execution support
+
+### Integration Features
+- OpenTelemetry integration for tracing and monitoring
+- Rich logging system with customizable formatters
+- DSPy integration for LLM interactions
+
+## Strengths
+
+1. **Simplicity**
+   - Clear separation of concerns
+   - Minimal boilerplate code
+   - Intuitive agent definition syntax
+
+2. **Flexibility**
+   - Support for multiple execution modes
+   - Extensible tool system
+   - Customizable output formatting
+
+3. **Production Readiness**
+   - Temporal integration for robust workflow execution
+   - Comprehensive error handling
+   - Built-in monitoring and tracing
+
+4. **Developer Experience**
+   - Clear documentation and examples
+   - Local debugging support
+   - Rich logging and error feedback
+
+## Areas for Potential Enhancement
+
+1. **Documentation**
+   - More comprehensive API documentation
+   - Additional usage examples
+   - Best practices guide
+
+2. **Testing Infrastructure**
+   - Expanded test coverage
+   - Testing utilities for agent development
+   - Performance benchmarking tools
+
+3. **Tool Ecosystem**
+   - Pre-built tool collections
+   - Tool documentation and examples
+   - Tool validation system
+
+## Conclusion
+
+Flock represents a sophisticated approach to agent-based LLM development, offering a powerful balance between simplicity and functionality. Its declarative model and robust architecture make it well-suited for both development and production environments. The framework's integration with Temporal and comprehensive feature set position it as a valuable tool for building complex LLM-powered applications.
+
+The framework's design choices, particularly its move away from traditional prompt engineering, demonstrate a forward-thinking approach to LLM application development. While there are areas for potential enhancement, the core architecture provides a solid foundation for future development and expansion.

flock_core-0.2.2/src/flock/core/context/context.py

@@ -0,0 +1,182 @@
+from dataclasses import asdict, dataclass, field
+from datetime import datetime
+from typing import Any, Literal
+
+from opentelemetry import trace
+
+from flock.core.context.context_vars import FLOCK_LAST_AGENT, FLOCK_LAST_RESULT
+from flock.core.logging.logging import get_logger
+from flock.core.util.serializable import Serializable
+
+logger = get_logger("context")
+tracer = trace.get_tracer(__name__)
+
+
+@dataclass
+class AgentRunRecord:
+    agent: str = field(default="")
+    data: dict[str, Any] = field(default_factory=dict)
+    timestamp: str = field(default="")
+    hand_off: dict = field(default_factory=dict)
+    called_from: str = field(default="")
+
+
+@dataclass
+class AgentDefinition:
+    agent_type: str = field(default="")
+    agent_name: str = field(default="")
+    agent_data: dict = field(default_factory=dict)
+    serializer: Literal["json", "cloudpickle", "msgpack"] = field(
+        default="cloudpickle"
+    )
+
+
+@dataclass
+class FlockContext(Serializable):
+    state: dict[str, Any] = field(default_factory=dict)
+    history: list[AgentRunRecord] = field(default_factory=list)
+    agent_definitions: dict[str, AgentDefinition] = field(default_factory=dict)
+    run_id: str = field(default="")
+    workflow_id: str = field(default="")
+    workflow_timestamp: str = field(default="")
+
+    def record(
+        self,
+        agent_name: str,
+        data: dict[str, Any],
+        timestamp: str,
+        hand_off: str,
+        called_from: str,
+    ) -> None:
+        record = AgentRunRecord(
+            agent=agent_name,
+            data=data.copy(),
+            timestamp=timestamp,
+            hand_off=hand_off,
+            called_from=called_from,
+        )
+        self.history.append(record)
+        for key, value in data.items():
+            self.set_variable(f"{agent_name}.{key}", value)
+        self.set_variable(FLOCK_LAST_RESULT, data)
+        self.set_variable(FLOCK_LAST_AGENT, agent_name)
+        logger.info(
+            "Agent run recorded",
+            agent=agent_name,
+            timestamp=timestamp,
+            data=data,
+        )
+        current_span = trace.get_current_span()
+        if current_span.get_span_context().is_valid:
+            current_span.add_event(
+                "record",
+                attributes={"agent": agent_name, "timestamp": timestamp},
+            )
+
+    def get_variable(self, key: str) -> Any:
+        return self.state.get(key)
+
+    def set_variable(self, key: str, value: Any) -> None:
+        old_value = self.state.get(key)
+        self.state[key] = value
+        if old_value != value:
+            logger.info(
+                "Context variable updated",
+                variable=key,
+                old=old_value,
+                new=value,
+            )
+            current_span = trace.get_current_span()
+            if current_span.get_span_context().is_valid:
+                current_span.add_event(
+                    "set_variable",
+                    attributes={
+                        "key": key,
+                        "old": str(old_value),
+                        "new": str(value),
+                    },
+                )
+
+    def deepcopy(self) -> "FlockContext":
+        return FlockContext.from_dict(self.to_dict())
+
+    def get_agent_history(self, agent_name: str) -> list[AgentRunRecord]:
+        return [record for record in self.history if record.agent == agent_name]
+
+    def next_input_for(self, agent) -> Any:
+        try:
+            if hasattr(agent, "input") and isinstance(agent.input, str):
+                keys = [k.strip() for k in agent.input.split(",") if k.strip()]
+                if len(keys) == 1:
+                    return self.get_variable(keys[0])
+                else:
+                    return {key: self.get_variable(key) for key in keys}
+            else:
+                return self.get_variable("init_input")
+        except Exception as e:
+            logger.error(
+                "Error getting next input for agent",
+                agent=agent.name,
+                error=str(e),
+            )
+            raise
+
+    def get_most_recent_value(self, variable_name: str) -> Any:
+        for history_record in reversed(self.history):
+            if variable_name in history_record.data:
+                return history_record.data[variable_name]
+
+    def get_agent_definition(self, agent_name: str) -> AgentDefinition | None:
+        return self.agent_definitions.get(agent_name)
+
+    def add_agent_definition(
+        self, agent_type: type, agent_name: str, agent_data: Any
+    ) -> None:
+        definition = AgentDefinition(
+            agent_type=agent_type.__name__,
+            agent_name=agent_name,
+            agent_data=agent_data,
+        )
+        self.agent_definitions[agent_name] = definition
+
+    # Use the reactive setter for dict-like access.
+    def __getitem__(self, key: str) -> Any:
+        return self.get_variable(key)
+
+    def __setitem__(self, key: str, value: Any) -> None:
+        self.set_variable(key, value)
+
+    def to_dict(self) -> dict[str, Any]:
+        def convert(obj):
+            if isinstance(obj, datetime):
+                return obj.isoformat()
+            if hasattr(obj, "__dataclass_fields__"):
+                return asdict(
+                    obj, dict_factory=lambda x: {k: convert(v) for k, v in x}
+                )
+            return obj
+
+        return convert(asdict(self))
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any]) -> "FlockContext":
+        def convert(obj):
+            if isinstance(obj, dict):
+                if "timestamp" in obj:
+                    return AgentRunRecord(
+                        **{
+                            **obj,
+                            "timestamp": datetime.fromisoformat(
+                                obj["timestamp"]
+                            ),
+                        }
+                    )
+                if "agent_type" in obj:
+                    return AgentDefinition(**obj)
+                return {k: convert(v) for k, v in obj.items()}
+            if isinstance(obj, list):
+                return [convert(v) for v in obj]
+            return obj
+
+        converted = convert(data)
+        return cls(**converted)