flock-core 0.4.0b38__tar.gz → 0.4.0b40__tar.gz

{flock_core-0.4.0b38 → flock_core-0.4.0b40}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: flock-core
-Version: 0.4.0b38
+Version: 0.4.0b40
 Summary: Declarative LLM Orchestration at Scale
 Author-email: Andre Ratzenberger <andre.ratzenberger@whiteduck.de>
 License-File: LICENSE

flock_core-0.4.0b40/docs/about/changelog.md

@@ -0,0 +1,37 @@
+---
+hide: # Optional: Hide table of contents on simple pages
+  - toc
+---
+
+# Changelog 📝
+
+All notable changes to **Flock** will be documented in this file.
+
+The format loosely follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and adheres to [Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+### Added
+- Core concepts & component documentation (this PR).
+- Placeholder replacement across docs.
+
+---
+
+## [0.1.0] – 2025-05-07 "Magpie"
+
+### Added
+- Initial public release to PyPI.
+- Declarative agent framework with evaluators, modules, routers.
+- Temporal execution path.
+- CLI & REST API wrappers.
+- Built-in tools: `get_web_content_as_markdown`, Azure Blob helper, markdown utils.
+
+### Changed
+- N/A
+
+### Fixed
+- N/A
+
+---
+
+*Older history prior to open-source release is maintained internally.*

flock_core-0.4.0b40/docs/about/contributing.md

@@ -0,0 +1,65 @@
+---
+hide: # Optional: Hide table of contents on simple pages
+  - toc
+---
+
+# Contributing Guide 🙌
+
+First off, thanks for taking the time to contribute!  Flock is still early but we value outside input.  We follow the [Contributor Covenant](https://www.contributor-covenant.org/) code of conduct.
+
+---
+
+## 1. Getting Started
+
+1. **Fork** the repo & create your feature branch (`git checkout -b feat/my-awesome-thing`).
+2. **Install** dev dependencies: `uv pip install -r requirements.txt && uv pip install -r requirements-dev.txt`.
+3. **Run tests**: `pytest -q` (they should pass before and after your change).
+4. **Make changes** – keep them atomic and well-documented.
+5. **Lint & type-check**: `ruff .` and `mypy src/`.
+6. **Commit** following *Conventional Commits* (`feat:`, `fix:`, `docs:` etc.).
+7. **Open a PR** against `main`.  Include a description, screenshots, and linked issues.
+
+---
+
+## 2. Docs Contributions
+
+Docs live under `docs/` and are built with **MkDocs Material**.
+
+```bash
+mkdocs serve  # live-reload at http://localhost:8000
+```
+
+* Use American English.
+* Keep sentences short; favour lists & tables.
+* Add code blocks with triple-backticks and language.
+
+---
+
+## 3. Coding Standards
+
+* Python `3.10+`.  Use type hints everywhere.
+* Follow `ruff` default rules + `black` formatting.
+* Keep imports sorted (`ruff format`).
+* Write **async-friendly** code (non-blocking I/O).
+
+---
+
+## 4. Tests
+
+* Place new tests in `tests/` mirroring the package path.
+* Use `pytest` fixtures instead of duplicating setup code.
+* For Temporal code, rely on the *Temporal Test Server* fixture.
+
+---
+
+## 5. Release Process
+
+1. Maintainer bumps version in `pyproject.toml` following *SemVer*.
+2. Changelog entry added in `docs/about/changelog.md`.
+3. `uv pip install -e .[all] && pytest`.
+4. `git tag vX.Y.Z && git push --tags`.
+5. GitHub Action publishes to PyPI.
+
+---
+
+Happy hacking! 🐦

flock_core-0.4.0b40/docs/components/evaluators.md

@@ -0,0 +1,82 @@
+---
+hide: # Optional: Hide table of contents on simple pages
+  - toc
+---
+
+# Evaluators 🧑‍⚖️
+
+Evaluators are responsible for **turning an agent's declarative specification into actual work**.  They live inside the agent instance (`agent.evaluator`) and must implement the async method:
+
+```python
+async def evaluate(self, agent: FlockAgent, inputs: dict, tools: list[Callable]) -> dict:  ...
+```
+
+The returned dict is validated against the agent's `output` signature.
+
+---
+
+## 1. Built-in Evaluators
+
+| Class | Location | Description |
+| ----- | -------- | ----------- |
+| `DeclarativeEvaluator` | `flock.evaluators.declarative` | Default.  Generates a prompt from the agent's signatures and calls `litellm`.
+| `DSPyEvaluator` | `flock.evaluators.dspy` | Integrates with [DSPy](https://github.com/stanfordnlp/dspy) for structured prompting & optimisation.
+| `RuleEvaluator` | `flock.evaluators.rule` | Pure-Python rules engine (no LLM). Useful for testing. |
+
+---
+
+## 2. Configuring Evaluators
+
+All evaluators accept a `*Config` dataclass (Pydantic model) with relevant fields.  Example for Declarative:
+
+```python
+from flock.evaluators.declarative import DeclarativeEvaluator, DeclarativeEvaluatorConfig
+
+config = DeclarativeEvaluatorConfig(
+    model="anthropic/claude-3-opus",
+    temperature=0.2,
+    max_tokens=2048,
+    stream=True,
+    include_thought_process=False,
+    use_cache=True,
+)
+
+evaluator = DeclarativeEvaluator(name="default", config=config)
+```
+
+You can then inject the evaluator when you instantiate the agent or replace it later:
+
+```python
+agent.evaluator = evaluator  # hot-swap!
+```
+
+---
+
+## 3. Writing a Custom Evaluator
+
+1. Subclass `FlockEvaluator` and optionally declare a `FlockEvaluatorConfig`.
+2. Implement `async evaluate(self, agent, inputs, tools)`.
+3. Register with the registry (optional) for serialization:
+
+```python
+from flock.core import flock_component
+
+@flock_component
+class MyCoolEvaluator(FlockEvaluator):
+    async def evaluate(self, agent, inputs, tools):
+        # do stuff…
+        return {"answer": "42"}
+```
+
+---
+
+## 4. Best Practices
+
+* Respect the `model` selected on the agent unless your evaluator has a reason to override.
+* Validate inputs early; raise `ValueError` to trigger `on_error` hooks.
+* Make expensive network calls **async** to keep the event loop responsive.
+* Return only the fields declared in the output signature (extra fields are dropped).
+
+---
+
+➡️ Continue to [Modules](modules.md) to see how to augment agent behaviour.

flock_core-0.4.0b40/docs/components/index.md

@@ -0,0 +1,43 @@
+---
+hide: # Optional: Hide table of contents on simple pages
+  - toc
+---
+
+# Components Overview 🧩
+
+Components are the **plug-in** architecture that powers extensibility in Flock.  They allow you to add behaviour without rewriting agent code.
+
+| Component Type | Purpose | Base Class |
+| -------------- | ------- | ---------- |
+| **Evaluator** | Implements an agent's core logic.  Often calls an LLM but could be pure Python. | `FlockEvaluator` |
+| **Module** | Hooks into the agent lifecycle for cross-cutting concerns (metrics, output formatting, memory, etc.). | `FlockModule` |
+| **Tool** | A single callable that an evaluator may invoke to access external functionality (web search, DB query, script execution…). | Registered via `@flock_tool` |
+
+Each component type is described in its own page:
+
+* [Evaluators](evaluators.md)
+* [Modules](modules.md)
+* [Tools](tools.md)
+
+---
+
+## Version Compatibility
+
+All core component base classes follow **Semantic Versioning**.  Adding new optional hook methods counts as a *minor* version bump, so custom components will keep working.
+
+---
+
+## Lifecycle Hook Matrix
+
+| Stage | Evaluator | Module |
+| ----- | --------- | ------ |
+| `initialize` | ❌ (Evaluator created beforehand) | ✅ |
+| `evaluate` | ✅ | ✅ (wrap/observe) |
+| `terminate` | ❌ | ✅ |
+| `on_error` | ❌ | ✅ |
+
+Tools do not participate in hooks; they are invoked directly by the evaluator.
+
+---
+
+Continue reading the dedicated pages for implementation details.

flock_core-0.4.0b40/docs/components/modules.md

@@ -0,0 +1,76 @@
+---
+hide: # Optional: Hide table of contents on simple pages
+  - toc
+---
+
+# Modules 🧩
+
+Modules let you **augment** agents with additional behaviour without touching their core logic.  They follow the classic *aspect-oriented* approach: hook into lifecycle stages and run code *around* the evaluator.
+
+---
+
+## 1. Anatomy
+
+```python
+class MyModule(FlockModule):
+    async def initialize(self, agent, inputs, context):
+        ...  # called before evaluate
+
+    async def after_evaluate(self, agent, result, context):
+        ...  # inspect/transform result
+
+    async def terminate(self, agent, context):
+        ...  # cleanup
+
+    async def on_error(self, agent, error, context):
+        ...  # exception handling
+```
+
+The base class also exposes a `.config` dataclass so users can set parameters declaratively.
+
+---
+
+## 2. Built-in Modules
+
+| Module | Purpose |
+| ------ | ------- |
+| `OutputModule` | Pretty-prints results to the console via *Rich*. |
+| `MetricsModule` | Records latency & token metrics, raises alerts. |
+| `MemoryModule` *(WIP)* | Persistent, vectorised memory per agent. |
+| `TracingModule` | Adds extra OpenTelemetry spans/tags. |
+
+You attach modules either during construction:
+
+```python
+agent = FlockFactory.create_default_agent(...)
+agent.add_module(OutputModule("output", OutputModuleConfig(render_table=True)))
+```
+
+Or pass them into `FlockAgent(modules={...})` directly.
+
+---
+
+## 3. Ordering
+
+Modules run in the order they were added.  Use this to your advantage (e.g. log metrics *after* output is printed).
+
+---
+
+## 4. Writing Your Own
+
+1. Subclass `FlockModule`.
+2. Implement any subset of the lifecycle hooks.
+3. Optionally create a `FlockModuleConfig` Pydantic model.
+4. Register via `@flock_component` so it serialises nicely.
+
+---
+
+## 5. Best Practices
+
+* Keep modules **stateless** or store state in the provided `context`.
+* Perform long-running or blocking work in the async hooks.
+* Use configurability (`*Config`) to avoid hard-coding behaviour.
+
+---
+
+Next up: [Tools](tools.md).

flock_core-0.4.0b40/docs/components/tools.md

@@ -0,0 +1,66 @@
+---
+hide: # Optional: Hide table of contents on simple pages
+  - toc
+---
+
+# Tools 🔧
+
+Tools are **single functions** that an evaluator can call to perform tasks the LLM cannot (or should not) handle by itself – e.g. web scraping, database queries, or calling another API.
+
+---
+
+## 1. Defining a Tool
+
+```python
+from flock.core import flock_tool
+
+@flock_tool
+def get_web_content_as_markdown(url: str) -> str:
+    """Fetch a web page and convert it to Markdown."""
+    import requests, markdownify  # lightweight example
+    html = requests.get(url, timeout=10).text
+    return markdownify.markdownify(html)
+```
+
+Decorating with `@flock_tool` registers the function in the **global registry** so any agent can reference it by symbol.
+
+---
+
+## 2. Using Tools in Agents
+
+```python
+agent = FlockFactory.create_default_agent(
+    name="page_analyser",
+    input="url: str",
+    output="title: str, headings: list[str]",
+    tools=[get_web_content_as_markdown],
+)
+```
+
+Inside your evaluator you simply call the function:
+
+```python
+content = await tools.get_web_content_as_markdown(inputs["url"])
+```
+
+The DeclarativeEvaluator automatically injects **function docs** into the prompt so the LLM knows when/how to call them.
+
+---
+
+## 3. Tool Signature Rules
+
+* Use **type hints** – they become part of the prompt + allow runtime validation.
+* Keep I/O **JSON-serialisable**; complex objects should be reduced to primitives.
+* Raise normal Python exceptions; they propagate to `on_error`.
+
+---
+
+## 4. Security Considerations
+
+* Be cautious when exposing file-system or shell operations.
+* Prefer *allow-lists* over *deny-lists* when building wrappers.
+* Use the `enable_temporal` sandbox if you need strict isolation.
+
+---
+
+That wraps up the components trilogy!  Head over to [Deployment](../deployment/index.md) to see how to ship your flock.

flock_core-0.4.0b40/docs/core-concepts/agents.md

@@ -0,0 +1,118 @@
+---
+hide: # Optional: Hide table of contents on simple pages
+  - toc
+---
+
+# Flock Agents 🐦
+
+A **FlockAgent** is the fundamental unit of work in the framework.  Each agent is *declarative* – you specify **what** goes in, **what** should come out, and (optionally) some high-level behavioural hints.  Flock takes care of turning that declaration into concrete logic at runtime.
+
+---
+
+## 1. Anatomy of an Agent
+
+```python
+from flock.core import FlockAgent, FlockFactory
+
+agent = FlockFactory.create_default_agent(
+    name="movie_pitcher",
+    description="Creates a fun movie idea for a given topic",
+    input="topic: str | The central subject of the movie",
+    output=(
+        "title: str | CAPSLOCK title, "
+        "runtime: int | Minutes, "
+        "synopsis: str | Over-the-top summary"
+    ),
+)
+```
+
+### Key Fields
+
+| Field | Type | Intent |
+| ----- | ---- | ------ |
+| `name` | `str` | Unique identifier; becomes the registry key. |
+| `model` | `str | None` | Override the default model for this agent. |
+| `description` | `str \| Callable` | High-level instructions; merged into the system prompt. |
+| `input` | `SignatureType` | Contract for accepted data. |
+| `output` | `SignatureType` | Contract for produced data. |
+| `tools` | `list[Callable]` | Extra callables the evaluator may invoke. |
+| `evaluator` | `FlockEvaluator` | Implementation of the agent's logic (defaults to DeclarativeEvaluator in `FlockFactory`). |
+| `handoff_router` | `FlockRouter` | Optional router that chooses the next agent. |
+| `modules` | `dict[str, FlockModule]` | Plug-in behaviours (metrics, memory, etc.). |
+
+All these fields are **Pydantic-validated** and fully serialisable via `Serializable`.
+
+---
+
+## 2. Signatures: Input & Output
+
+Signatures are written in a compact mini-DSL:
+
+* `field` – just a name (type & description inferred by the LLM).
+* `field: type` – adds a type hint.
+* `field: type | description` – adds a natural-language description.
+* Multiple fields are comma-separated.
+* Lists/dicts follow normal Python typing: `list[dict[str, str]]`.
+
+Alternatively, pass a **Pydantic model** for static typing:
+
+```python
+from pydantic import BaseModel
+
+class SearchIn(BaseModel):
+    query: str
+    top_k: int = 5
+
+class SearchOut(BaseModel):
+    documents: list[str]
+
+search_agent = FlockAgent(
+    name="searcher",
+    input=SearchIn,
+    output=SearchOut,
+    evaluator=CustomSearchEvaluator(...),
+)
+```
+
+---
+
+## 3. Lifecycle
+
+```mermaid
+flowchart LR
+    A[initialize] --> B[evaluate]
+    B --> C[terminate]
+    B -. error .-> D[on_error]
+```
+
+1. **initialize** – prepare resources (DB connection, load embeddings, etc.).
+2. **evaluate** – main logic executed by the evaluator.
+3. **terminate** – clean-up, persist metrics.
+4. **on_error** – triggered if any previous stage raises.
+
+Modules can hook into each stage to extend behaviour.
+
+---
+
+## 4. Attaching Modules & Routers
+
+```python
+from flock.modules.output import OutputModule, OutputModuleConfig
+from flock.routers.agent import AgentRouter, AgentRouterConfig
+
+agent.add_module(OutputModule("output", OutputModuleConfig(render_table=True)))
+agent.handoff_router = AgentRouter("router", AgentRouterConfig())
+```
+
+---
+
+## 5. Best Practices
+
+* Keep `description` concise; use the signature for fine-grained control.
+* Prefer Pydantic models for complex schemas – you get validation for free.
+* Separate concerns: evaluator handles *logic*, modules handle *cross-cutting* tasks.
+* Register reusable tools with `@flock_tool` so any agent can adopt them.
+
+---
+
+**Next:**  Learn *why* this declarative approach works in [Declarative Programming](declarative.md).

flock_core-0.4.0b40/docs/core-concepts/declarative.md

@@ -0,0 +1,121 @@
+---
+hide: # Optional: Hide table of contents on simple pages
+  - toc
+---
+
+# Declarative Programming in Flock ✨
+
+Most frameworks ask you to write long, brittle *prompts* or imperative code that walks the model through every step.  **Flock** flips this on its head: you describe the *contract*, not the procedure.  The evaluator turns that contract into the right LLM calls at run-time.
+
+---
+
+## 1. Why Declarative?
+
+1. **Clarity** – A two-line signature communicates intent faster than a 200-line prompt.
+2. **Reusability** – When you change models (say GPT-4 → Claude) you rarely touch the agent spec.
+3. **Validation** – Flock can automatically type-check inputs/outputs, catch errors early, and generate OpenAPI docs.
+4. **Optimisation** – The framework can inject extra instructions (few-shot examples, chain-of-thought, etc.) without polluting user code.
+
+---
+
+## 2. The Mini-DSL
+
+Signatures are plain strings with optional type hints & descriptions:
+
+```text
+"query: str | The search query, top_k: int | # documents to retrieve"
+```
+
+Behind the scenes Flock parses this into a structured schema and merges it into a **system prompt** similar to:
+
+```text
+You are `search_agent`.
+You must respond with a JSON object matching the following schema:
+{
+  "documents": list[str]
+}
+
+Guidelines:
+- The `documents` field must contain exactly `top_k` results.
+- ...additional instructions from modules/evaluator...
+```
+
+The spec is also turned into **Pydantic** models at runtime so the response is validated before it reaches your code.
+
+---
+
+## 3. Dynamic Signatures
+
+Sometimes the shape depends on runtime settings.  Provide a **callable** that returns a signature string or model:
+
+```python
+from datetime import date
+
+def invoice_out():
+    year = date.today().year
+    return (
+        f"number: str | Invoice number {year}-****, "
+        "amount: float, "
+        "currency: str, "
+        "items: list[str]"
+    )
+
+agent = FlockAgent(
+    name="invoicer",
+    input="order_id: str",
+    output=invoice_out,
+    evaluator=InvoiceEvaluator(...),
+)
+```
+
+---
+
+## 4. Going Fully-Typed with Pydantic
+
+If you prefer proper Python types (IDE autocompletion, mypy, etc.), define models:
+
+```python
+class OrderIn(BaseModel):
+    order_id: str
+    include_discount: bool = False
+
+class InvoiceOut(BaseModel):
+    number: str
+    amount: float
+    currency: str
+    items: list[str]
+
+agent = FlockAgent(
+    name="invoicer",
+    input=OrderIn,
+    output=InvoiceOut,
+    evaluator=InvoiceEvaluator(...),
+)
+```
+
+Flock will **not** expose these classes to the LLM (that would leak Python internals).  Instead it converts them to an equivalent JSON schema.
+
+---
+
+## 5. Prompt Composition Pipeline
+
+1. **Base system prompt** – set by evaluator (`DeclarativeEvaluator` by default).
+2. **Agent description** – `agent.description` text.
+3. **Input / output schema** – rendered as JSON spec + additional guidance.
+4. **Module injections** – e.g. `OutputModule` may request thought-process streaming.
+5. **Tool docs** – For each tool, a short description and parameter list is appended.
+
+The final prompt is cached (unless `use_cache=False`) for efficiency.
+
+---
+
+## 6. Limits & Tips
+
+* Keep field names **snake_case** – easier to parse.
+* Large nested objects are fine but remember token costs.
+* Use `include_thought_process=True` in `DeclarativeEvaluatorConfig` if you need chain-of-thought output.
+* The more explicit the types/descriptions, the more reliable the model's output.
+
+---
+
+**Next:**  See how agents chain together in [Workflows](workflows.md).