flock-core 0.3.15__tar.gz → 0.3.17__tar.gz

{flock_core-0.3.15 → flock_core-0.3.17}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: flock-core
-Version: 0.3.15
+Version: 0.3.17
 Summary: Declarative LLM Orchestration at Scale
 Author-email: Andre Ratzenberger <andre.ratzenberger@whiteduck.de>
 License-File: LICENSE
@@ -8,6 +8,7 @@ Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Requires-Python: >=3.10
+Requires-Dist: azure-search-documents>=11.5.2
 Requires-Dist: chromadb>=0.6.3
 Requires-Dist: cloudpickle>=3.1.1
 Requires-Dist: devtools>=0.12.2
@@ -29,6 +30,7 @@ Requires-Dist: prometheus-client>=0.21.1
 Requires-Dist: pydantic>=2.10.5
 Requires-Dist: python-box>=7.3.2
 Requires-Dist: python-decouple>=3.8
+Requires-Dist: python-dotenv>=1.0.1
 Requires-Dist: questionary>=2.1.0
 Requires-Dist: rich>=13.9.4
 Requires-Dist: sentence-transformers>=3.4.1

flock_core-0.3.17/docs/core-concepts/tools/azure-tools.md

@@ -0,0 +1,221 @@
+# Azure Search Tools
+
+Flock provides a set of tools for seamless integration with [Azure AI Search](https://learn.microsoft.com/en-us/azure/search/), allowing you to leverage powerful vector search and document retrieval capabilities in your AI agents.
+
+## Overview
+
+Azure AI Search (formerly Azure Cognitive Search) is a cloud search service that gives developers APIs and tools for building rich search experiences over private, heterogeneous content in web, mobile, and enterprise applications. In Flock, we've provided tools to easily interact with Azure AI Search, allowing your agents to store, search, and retrieve information using both text and vector embeddings.
+
+## Features
+
+- Create and manage search indexes
+- Upload and manage documents
+- Perform text-based and vector-based searches
+- Retrieve detailed index statistics
+- Delete documents and manage content
+
+## Environment Configuration
+
+The Azure Search tools rely on environment variables for configuration:
+
+```python
+# Required environment variables
+AZURE_SEARCH_ENDPOINT  # The endpoint URL for your Azure AI Search service
+AZURE_SEARCH_API_KEY   # The API key for your Azure AI Search service
+AZURE_SEARCH_INDEX_NAME  # Optional: A default index name to use
+```
+
+## Available Tools
+
+### Initialization
+
+```python
+from flock.core.tools.azure_tools import azure_search_initialize_clients
+
+# Initialize clients (uses environment variables by default)
+clients = azure_search_initialize_clients()
+
+# Or specify endpoints manually
+clients = azure_search_initialize_clients(
+    endpoint="https://your-service.search.windows.net",
+    api_key="your-api-key",
+    index_name="your-index-name"
+)
+```
+
+### Index Management
+
+```python
+from flock.core.tools.azure_tools import (
+    azure_search_create_index,
+    azure_search_list_indexes,
+    azure_search_get_index_statistics
+)
+from azure.search.documents.indexes.models import (
+    SearchField,
+    SearchFieldDataType,
+    SimpleField,
+    SearchableField
+)
+
+# Create fields for the index
+fields = [
+    SimpleField(name="id", type=SearchFieldDataType.String, key=True),
+    SearchableField(name="title", type=SearchFieldDataType.String),
+    SearchableField(name="content", type=SearchFieldDataType.String)
+]
+
+# Create index
+result = azure_search_create_index(
+    index_name="my-index",
+    fields=fields
+)
+
+# List all indexes
+indexes = azure_search_list_indexes()
+
+# Get index statistics
+stats = azure_search_get_index_statistics(index_name="my-index")
+```
+
+### Vector Search Index Creation
+
+```python
+from flock.core.tools.azure_tools import azure_search_create_vector_index
+
+# Create a vector index
+fields = [
+    {"name": "id", "type": "string", "key": True},
+    {"name": "title", "type": "string", "searchable": True},
+    {"name": "content", "type": "string", "searchable": True},
+    {"name": "embedding", "type": "collection", "vector": True}
+]
+
+result = azure_search_create_vector_index(
+    fields=fields,
+    vector_dimensions=1536,  # Dimensions for your vector embeddings
+    index_name="vector-index",
+    algorithm_kind="hnsw"  # Using HNSW algorithm for vector search
+)
+```
+
+### Document Management
+
+```python
+from flock.core.tools.azure_tools import (
+    azure_search_upload_documents,
+    azure_search_get_document,
+    azure_search_delete_documents
+)
+
+# Upload documents
+documents = [
+    {
+        "id": "doc1",
+        "title": "Sample Document",
+        "content": "This is a sample document for Azure Search."
+    },
+    {
+        "id": "doc2",
+        "title": "Another Document",
+        "content": "This is another sample document."
+    }
+]
+
+upload_result = azure_search_upload_documents(
+    documents=documents,
+    index_name="my-index"
+)
+
+# Get a specific document
+document = azure_search_get_document(
+    key="doc1",
+    index_name="my-index"
+)
+
+# Delete documents
+delete_result = azure_search_delete_documents(
+    keys=["doc1", "doc2"],
+    index_name="my-index"
+)
+```
+
+### Search and Query
+
+```python
+from flock.core.tools.azure_tools import azure_search_query
+
+# Text-based search
+results = azure_search_query(
+    search_text="sample",
+    index_name="my-index"
+)
+
+# Filter-based search
+filtered_results = azure_search_query(
+    filter="id eq 'doc1'",
+    index_name="my-index"
+)
+
+# Vector search (for vector indexes)
+vector = [0.1, 0.2, 0.3, ...]  # Your vector embedding
+vector_results = azure_search_query(
+    vector=vector,
+    vector_field="embedding",
+    vector_k=10,  # Return top 10 results
+    index_name="vector-index"
+)
+```
+
+## Integration with Agents
+
+Azure Search tools can be easily integrated with your Flock agents to provide powerful retrieval capabilities:
+
+```python
+from flock import Flock, FlockFactory
+from flock.core.tools.azure_tools import (
+    azure_search_query,
+    azure_search_upload_documents
+)
+
+# Create an agent with access to Azure Search tools
+agent = FlockFactory.create_default_agent(
+    name="search_agent",
+    input="search_query",
+    output="search_results",
+    tools=[azure_search_query]
+)
+
+# Create a document management agent
+upload_agent = FlockFactory.create_default_agent(
+    name="upload_agent",
+    input="documents_to_upload",
+    output="upload_status",
+    tools=[azure_search_upload_documents]
+)
+
+# Add to Flock
+flock = Flock()
+flock.add_agent(agent)
+flock.add_agent(upload_agent)
+```
+
+## Best Practices
+
+1. **Environment Variables**: Use environment variables for sensitive information like API keys.
+2. **Error Handling**: Implement proper error handling when using these tools, as network requests can sometimes fail.
+3. **Batching**: When uploading large numbers of documents, consider batching them in groups of 100-1000.
+4. **Indexing Delay**: Allow a small delay (1-2 seconds) after uploading documents before searching them to ensure indexing is complete.
+5. **Vector Dimensions**: Ensure your vector dimensions match between your embeddings and the index configuration.
+
+## Limitations
+
+- The maximum number of documents that can be uploaded in a single request is 1000.
+- Vector search requires the appropriate SKU tier in Azure AI Search.
+- Certain operations may have rate limits depending on your Azure service tier.
+
+## Further Resources
+
+- [Azure AI Search Documentation](https://docs.microsoft.com/en-us/azure/search/)
+- [Vector Search in Azure AI Search](https://learn.microsoft.com/en-us/azure/search/vector-search-overview)
+- [Azure Python SDK Documentation](https://docs.microsoft.com/en-us/python/api/overview/azure/search-documents-readme?view=azure-python) 

flock_core-0.3.17/docs/core-concepts/tools/overview.md

@@ -0,0 +1,96 @@
+# Tools Overview
+
+Flock provides a variety of tools that agents can use to interact with external systems, access specialized capabilities, and extend their functionality beyond language processing.
+
+## What Are Tools?
+
+In Flock, tools are functions that agents can invoke to perform specific tasks. Unlike the agent's core language capabilities, tools allow agents to:
+
+- Access external data sources and APIs
+- Perform specialized calculations or operations
+- Interact with services like databases, search engines, or vector stores
+- Execute custom business logic
+
+Tools are designed to be modular and composable, allowing you to give your agents exactly the capabilities they need for their specific tasks.
+
+## Available Tool Categories
+
+Flock includes several categories of built-in tools:
+
+1. **Search and Retrieval Tools**
+   - Azure AI Search tools for document and vector search
+   - (More coming soon)
+
+2. **Utility Tools**
+   - Math operations
+   - Data processing
+   - File operations
+
+3. **Integration Tools**
+   - External API connectors
+   - Service integrations
+
+## Using Tools with Agents
+
+Tools can be assigned to agents during creation:
+
+```python
+from flock import FlockFactory
+from flock.core.tools.azure_tools import azure_search_query
+
+# Create an agent with access to the Azure Search query tool
+agent = FlockFactory.create_default_agent(
+    name="search_agent",
+    input="query",
+    output="results",
+    tools=[azure_search_query]
+)
+```
+
+Agents can then use these tools during their execution cycle, calling them as needed to fulfill their tasks.
+
+## Creating Custom Tools
+
+You can easily create custom tools for your specific needs:
+
+```python
+from flock.core.logging.trace_and_logged import traced_and_logged
+
+@traced_and_logged
+def my_custom_tool(param1: str, param2: int = 0) -> dict:
+    """
+    A custom tool that performs a specific task.
+    
+    Args:
+        param1: The first parameter
+        param2: The second parameter with default value
+        
+    Returns:
+        A dictionary with the results
+    """
+    # Your tool implementation here
+    result = {"status": "success", "value": f"{param1} processed with {param2}"}
+    return result
+```
+
+Key points for creating effective tools:
+
+1. Use the `@traced_and_logged` decorator for proper tracing and logging
+2. Provide clear type hints for parameters and return values
+3. Write descriptive docstrings explaining what the tool does
+4. Handle errors gracefully within the tool implementation
+5. Return structured data that agents can easily process
+
+## Tool Best Practices
+
+1. **Error Handling**: Implement proper error handling within tools to prevent agent failures
+2. **Tool Composition**: Design tools to be composable so they can be combined for complex workflows
+3. **Parameter Validation**: Validate parameters within tools to prevent incorrect usage
+4. **Documentation**: Clearly document what each tool does, its parameters, and return values
+5. **Security**: Be mindful of security implications, especially for tools that access external systems
+
+## Next Steps
+
+- Explore specific tool implementations in detail in their respective documentation pages
+- Learn how to create your own custom tools tailored to your specific use cases
+- See examples of tools in action in the Tutorials section 

{flock_core-0.3.15 → flock_core-0.3.17}/docs/getting-started/concepts.md

@@ -83,6 +83,8 @@ web_agent = FlockFactory.create_default_agent(
 )
 ```
 
+For more advanced capabilities, Flock provides specialized tools like [Azure Search integration](../core-concepts/tools/azure-tools.md) for powerful vector search and document retrieval. Check out the [Tools Overview](../core-concepts/tools/overview.md) for a comprehensive list of available tools.
+
 ### Modules
 
 Modules add capabilities to your agents through lifecycle hooks:

{flock_core-0.3.15 → flock_core-0.3.17}/docs/release-notes/release_notes.md

@@ -71,6 +71,7 @@ Like a hummingbird, modules are small and nimble code packages. Put enough of th
 - **Output Module** – Advanced output formatting and storage  
 - **Metrics Module** – Detailed performance tracking  
 - **Zep Module** – Uses Zep for Knowledge Graphs
+- **Azure Search Tools** – Integration with Azure AI Search for vector search and document retrieval
 
 ---
 

flock_core-0.3.17/examples/02_concepts/context/context.py

@@ -0,0 +1,57 @@
+
+from flock.core import Flock, FlockFactory
+from flock.core.flock_module import FlockModule, FlockModuleConfig
+from flock.routers.default.default_router import DefaultRouter, DefaultRouterConfig
+
+class ContextModule(FlockModule):
+    def terminate(self, agent, inputs, result, context=None):
+        context.set_variable("flock_agent1.a_random_name", "John Doe")
+     
+
+flock = Flock()
+
+
+
+# Flock has an advanced context system that allows you to store and retrieve data
+# across agents. The context is a dictionary that can be accessed and modified by 
+# all agents in the flock via modules
+# Agents will write their inputs and outputs to the context prefixed with "agentname."
+
+
+
+flock_agent_1 = FlockFactory.create_default_agent(
+    name="flock_agent1",
+    input="",
+    output="a_random_name: str",
+)
+
+
+flock_agent_2 = FlockFactory.create_default_agent(
+    name="flock_agent2",
+    input="a_random_name",
+    output="name_in_caps: str",
+)
+flock_agent_2.add_module(ContextModule(name="context_module", config= FlockModuleConfig()))
+
+# Agent3 will reverse John Doe to eoD nhoJ
+flock_agent_3 = FlockFactory.create_default_agent(
+    name="flock_agent3",
+    input="flock_agent1.a_random_name",
+    output="name_reversed: str",
+    wait_for_input=True,
+    print_context=True,
+)
+
+
+flock_agent_1.handoff_router = DefaultRouter(config=DefaultRouterConfig(hand_off=flock_agent_2.name))
+flock_agent_2.handoff_router = DefaultRouter(config=DefaultRouterConfig(hand_off=flock_agent_3.name))
+flock_agent_3.handoff_router = DefaultRouter(config=DefaultRouterConfig(hand_off=flock_agent_1.name))
+flock.add_agent(flock_agent_1)
+flock.add_agent(flock_agent_2)
+flock.add_agent(flock_agent_3)
+
+flock.run(
+    input={},
+    start_agent=flock_agent_1,
+    agents=[flock_agent_1, flock_agent_2, flock_agent_3]
+)

flock_core-0.3.17/examples/02_concepts/tools/azure_tools_example.py

@@ -0,0 +1,221 @@
+"""
+Azure AI Search Tools Example with Flock Agent Framework
+
+This example demonstrates how to use Azure AI Search tools within the Flock agent framework.
+It creates agents that can:
+1. Create a search index
+2. Upload documents to the index
+3. Query the index with both text and filter criteria
+4. Process and summarize search results
+
+Requirements:
+- Set the following environment variables:
+  - AZURE_SEARCH_ENDPOINT: Your Azure AI Search service endpoint URL
+  - AZURE_SEARCH_API_KEY: Your Azure AI Search API key
+"""
+
+import os
+import time
+import uuid
+import json
+from dotenv import load_dotenv
+
+from flock.core import Flock, FlockFactory
+from flock.core.logging.formatters.themes import OutputTheme
+from flock.core.tools.azure_tools import (
+    azure_search_initialize_clients,
+    azure_search_create_index,
+    azure_search_upload_documents,
+    azure_search_query,
+    azure_search_get_document,
+    azure_search_delete_documents,
+    azure_search_list_indexes,
+    azure_search_get_index_statistics
+)
+from flock.core.tools import basic_tools
+
+# Load environment variables from .env file
+load_dotenv()
+
+# Make sure Azure Search credentials are set
+if not os.environ.get("AZURE_SEARCH_ENDPOINT") or not os.environ.get("AZURE_SEARCH_API_KEY"):
+    raise ValueError(
+        "Please set AZURE_SEARCH_ENDPOINT and AZURE_SEARCH_API_KEY environment variables"
+    )
+
+# Create a unique index name for this example
+timestamp = int(time.time())
+random_id = str(uuid.uuid4())[:8]
+INDEX_NAME = f"flock-example-{timestamp}-{random_id}"
+os.environ["AZURE_SEARCH_INDEX_NAME"] = INDEX_NAME
+
+# --------------------------------
+# Create the flock
+# --------------------------------
+flock = Flock()
+
+# --------------------------------
+# Define our agents
+# --------------------------------
+
+# 1. Index Creator Agent
+# This agent creates a search index with appropriate schema
+index_creator = FlockFactory.create_default_agent(
+    name="index_creator",
+    input="index_requirements",
+    output="created_index_details",
+    tools=[
+        azure_search_create_index,
+        basic_tools.code_eval,
+        basic_tools.json_parse_safe
+    ],
+    enable_rich_tables=True,
+    output_theme=OutputTheme.adventuretime,
+)
+
+# 2. Document Uploader Agent
+# This agent uploads sample documents to the search index
+document_uploader = FlockFactory.create_default_agent(
+    name="document_uploader",
+    input="created_index_details, documents_to_upload",
+    output="upload_results",
+    tools=[
+        azure_search_upload_documents,
+        basic_tools.code_eval,
+        basic_tools.json_parse_safe
+    ],
+    enable_rich_tables=True,
+    output_theme=OutputTheme.adventuretime,
+)
+
+# 3. Search Agent
+# This agent searches the index and processes results
+search_agent = FlockFactory.create_default_agent(
+    name="search_agent",
+    input="upload_results, search_query, filter_criteria",
+    output="search_results: dict, relevant_documents: list, summary: str",
+    tools=[
+        azure_search_query,
+        azure_search_get_document,
+        basic_tools.code_eval,
+        basic_tools.json_parse_safe
+    ],
+    enable_rich_tables=True,
+    output_theme=OutputTheme.adventuretime,
+)
+
+# 4. Cleanup Agent (optional)
+# This agent gets statistics and can delete documents if needed
+cleanup_agent = FlockFactory.create_default_agent(
+    name="cleanup_agent",
+    input="search_results",
+    output="index_statistics: dict, cleanup_recommendation: str",
+    tools=[
+        azure_search_get_index_statistics,
+        azure_search_list_indexes,
+        azure_search_delete_documents,
+        basic_tools.code_eval
+    ],
+    enable_rich_tables=True,
+    output_theme=OutputTheme.adventuretime,
+)
+
+# --------------------------------
+# Configure agent flow
+# --------------------------------
+# Set up the agent sequence:
+# index_creator -> document_uploader -> search_agent -> cleanup_agent
+from flock.routers.default.default_router import DefaultRouter, DefaultRouterConfig
+
+index_creator.handoff_router = DefaultRouter(config=DefaultRouterConfig(hand_off=document_uploader.name))
+document_uploader.handoff_router = DefaultRouter(config=DefaultRouterConfig(hand_off=search_agent.name))
+search_agent.handoff_router = DefaultRouter(config=DefaultRouterConfig(hand_off=cleanup_agent.name))
+
+# Add all agents to the flock
+flock.add_agent(index_creator)
+flock.add_agent(document_uploader)
+flock.add_agent(search_agent)
+flock.add_agent(cleanup_agent)
+
+# --------------------------------
+# Define sample data
+# --------------------------------
+# Define our sample documents
+sample_documents = [
+    {
+        "id": "doc1",
+        "title": "Azure AI Search Overview",
+        "content": "Azure AI Search is a cloud search service with built-in AI capabilities.",
+        "category": "cloud",
+        "rating": 5
+    },
+    {
+        "id": "doc2",
+        "title": "Python Development Best Practices",
+        "content": "Write clean, maintainable Python code by following established best practices.",
+        "category": "development",
+        "rating": 4
+    },
+    {
+        "id": "doc3",
+        "title": "Azure and Python Integration",
+        "content": "Learn how to use Azure services with Python applications effectively.",
+        "category": "cloud",
+        "rating": 5
+    },
+    {
+        "id": "doc4",
+        "title": "Search Engine Optimization",
+        "content": "Improve your website's visibility in search engine results pages.",
+        "category": "marketing",
+        "rating": 3
+    },
+    {
+        "id": "doc5",
+        "title": "Machine Learning on Azure",
+        "content": "Train and deploy machine learning models using Azure ML.",
+        "category": "ai",
+        "rating": 5
+    }
+]
+
+# Define index requirements
+index_requirements = """
+Create a search index with the following fields:
+- id: string, key field
+- title: string, searchable, sortable
+- content: string, searchable
+- category: string, filterable
+- rating: integer, filterable, sortable
+
+The index name should be dynamically set from the AZURE_SEARCH_INDEX_NAME environment variable.
+"""
+
+# --------------------------------
+# Run the flock
+# --------------------------------
+# Start the agent workflow
+flock.run(
+    start_agent=index_creator,
+    input={
+        "index_requirements": index_requirements,
+        "documents_to_upload": json.dumps(sample_documents),
+        "search_query": "Azure cloud capabilities",
+        "filter_criteria": "rating eq 5"
+    }
+)
+
+# --------------------------------
+# Cleanup
+# --------------------------------
+# Delete the test index after running the example
+print(f"\nCleaning up: Deleting index '{INDEX_NAME}'...")
+try:
+    clients = azure_search_initialize_clients()
+    index_client = clients["index_client"]
+    index_client.delete_index(INDEX_NAME)
+    print(f"Index '{INDEX_NAME}' deleted successfully.")
+except Exception as e:
+    print(f"Warning: Failed to delete test index '{INDEX_NAME}': {e}")
+
+print("\nExample completed!")