flock-core 0.5.21__py3-none-any.whl → 0.5.23__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of flock-core might be problematic. Click here for more details.
- flock/api/models.py +3 -2
- flock/api/service.py +0 -1
- flock/core/agent.py +51 -16
- flock/core/orchestrator.py +18 -6
- flock/core/subscription.py +151 -8
- flock/semantic/__init__.py +49 -0
- flock/semantic/context_provider.py +173 -0
- flock/semantic/embedding_service.py +239 -0
- flock_core-0.5.23.dist-info/METADATA +976 -0
- {flock_core-0.5.21.dist-info → flock_core-0.5.23.dist-info}/RECORD +13 -10
- flock_core-0.5.21.dist-info/METADATA +0 -1327
- {flock_core-0.5.21.dist-info → flock_core-0.5.23.dist-info}/WHEEL +0 -0
- {flock_core-0.5.21.dist-info → flock_core-0.5.23.dist-info}/entry_points.txt +0 -0
- {flock_core-0.5.21.dist-info → flock_core-0.5.23.dist-info}/licenses/LICENSE +0 -0
|
@@ -1,1327 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: flock-core
|
|
3
|
-
Version: 0.5.21
|
|
4
|
-
Summary: Flock: A declrative framework for building and orchestrating AI agents.
|
|
5
|
-
Author-email: Andre Ratzenberger <andre.ratzenberger@whiteduck.de>
|
|
6
|
-
License: MIT
|
|
7
|
-
License-File: LICENSE
|
|
8
|
-
Requires-Python: >=3.12
|
|
9
|
-
Requires-Dist: aiosqlite>=0.20.0
|
|
10
|
-
Requires-Dist: devtools>=0.12.2
|
|
11
|
-
Requires-Dist: dspy==3.0.3
|
|
12
|
-
Requires-Dist: duckdb>=1.1.0
|
|
13
|
-
Requires-Dist: fastapi>=0.117.1
|
|
14
|
-
Requires-Dist: hanging-threads>=2.0.7
|
|
15
|
-
Requires-Dist: httpx>=0.28.1
|
|
16
|
-
Requires-Dist: litellm==1.78.0
|
|
17
|
-
Requires-Dist: loguru>=0.7.3
|
|
18
|
-
Requires-Dist: mcp>=1.7.1
|
|
19
|
-
Requires-Dist: opentelemetry-api>=1.30.0
|
|
20
|
-
Requires-Dist: opentelemetry-exporter-jaeger-proto-grpc>=1.21.0
|
|
21
|
-
Requires-Dist: opentelemetry-exporter-jaeger>=1.21.0
|
|
22
|
-
Requires-Dist: opentelemetry-exporter-otlp>=1.30.0
|
|
23
|
-
Requires-Dist: opentelemetry-instrumentation-logging>=0.51b0
|
|
24
|
-
Requires-Dist: opentelemetry-sdk>=1.30.0
|
|
25
|
-
Requires-Dist: poethepoet>=0.30.0
|
|
26
|
-
Requires-Dist: pydantic[email]>=2.11.9
|
|
27
|
-
Requires-Dist: rich>=14.1.0
|
|
28
|
-
Requires-Dist: toml>=0.10.2
|
|
29
|
-
Requires-Dist: typer>=0.19.2
|
|
30
|
-
Requires-Dist: uvicorn>=0.37.0
|
|
31
|
-
Requires-Dist: websockets>=15.0.1
|
|
32
|
-
Description-Content-Type: text/markdown
|
|
33
|
-
|
|
34
|
-
<p align="center">
|
|
35
|
-
<img alt="Flock Banner" src="docs/assets/images/flock.png" width="800">
|
|
36
|
-
</p>
|
|
37
|
-
<p align="center">
|
|
38
|
-
<a href="https://whiteducksoftware.github.io/flock/" target="_blank"><img alt="Documentation" src="https://img.shields.io/badge/docs-online-blue?style=for-the-badge&logo=readthedocs"></a>
|
|
39
|
-
<a href="https://pypi.org/project/flock-core/" target="_blank"><img alt="PyPI Version" src="https://img.shields.io/pypi/v/flock-core?style=for-the-badge&logo=pypi&label=pip%20version"></a>
|
|
40
|
-
<img alt="Python Version" src="https://img.shields.io/badge/python-3.12%2B-blue?style=for-the-badge&logo=python">
|
|
41
|
-
<a href="LICENSE" target="_blank"><img alt="License" src="https://img.shields.io/github/license/whiteducksoftware/flock?style=for-the-badge"></a>
|
|
42
|
-
<a href="https://whiteduck.de" target="_blank"><img alt="Built by white duck" src="https://img.shields.io/badge/Built%20by-white%20duck%20GmbH-white?style=for-the-badge&labelColor=black"></a>
|
|
43
|
-
<a href="https://codecov.io/gh/whiteducksoftware/flock" target="_blank"><img alt="Test Coverage" src="https://codecov.io/gh/whiteducksoftware/flock/branch/main/graph/badge.svg?token=YOUR_TOKEN_HERE&style=for-the-badge"></a>
|
|
44
|
-
<img alt="Tests" src="https://img.shields.io/badge/tests-1300+-brightgreen?style=for-the-badge">
|
|
45
|
-
</p>
|
|
46
|
-
|
|
47
|
-
---
|
|
48
|
-
|
|
49
|
-
# Flock 0.5: Declarative Blackboard Multi-Agent Orchestration
|
|
50
|
-
|
|
51
|
-
> **Stop engineering prompts. Start declaring contracts.**
|
|
52
|
-
|
|
53
|
-
Flock is a production-focused framework for orchestrating AI agents through **declarative type contracts** and **blackboard architecture**—proven patterns from distributed systems, decades of experience with microservice architectures, and classical AI—now applied to modern LLMs.
|
|
54
|
-
|
|
55
|
-
**📖 [Read the full documentation →](https://whiteducksoftware.github.io/flock)**
|
|
56
|
-
|
|
57
|
-
**Quick links:**
|
|
58
|
-
- **[Getting Started](https://whiteducksoftware.github.io/flock/getting-started/installation/)** - Installation and first steps
|
|
59
|
-
- **[Tutorials](https://whiteducksoftware.github.io/flock/tutorials/)** - Step-by-step learning path
|
|
60
|
-
- [Custom Engines: Emoji Vibes & Batch Brews](https://whiteducksoftware.github.io/flock/tutorials/custom-engines/)
|
|
61
|
-
- [Custom Agent Components: Foreshadow & Hype](https://whiteducksoftware.github.io/flock/tutorials/custom-agent-components/)
|
|
62
|
-
- **[User Guides](https://whiteducksoftware.github.io/flock/guides/)** - In-depth feature documentation
|
|
63
|
-
- **[API Reference](https://whiteducksoftware.github.io/flock/reference/api/)** - Complete API documentation
|
|
64
|
-
- **[Roadmap](https://whiteducksoftware.github.io/flock/about/roadmap/)** - What's coming in v1.0
|
|
65
|
-
- **Architecture & Patterns:**
|
|
66
|
-
- [Architecture Overview](docs/architecture.md) - System design and module organization
|
|
67
|
-
- [Error Handling Patterns](docs/patterns/error_handling.md) - Production error handling guide
|
|
68
|
-
- [Async Patterns](docs/patterns/async_patterns.md) - Async/await best practices
|
|
69
|
-
|
|
70
|
-
---
|
|
71
|
-
|
|
72
|
-
## The Problem With Current Approaches
|
|
73
|
-
|
|
74
|
-
Building production multi-agent systems today means dealing with:
|
|
75
|
-
|
|
76
|
-
**🔥 Prompt Engineering Hell**
|
|
77
|
-
```python
|
|
78
|
-
|
|
79
|
-
prompt = """You are an expert code reviewer. When you receive code, you should...
|
|
80
|
-
[498 more lines of instructions that the LLM ignores half the time]"""
|
|
81
|
-
|
|
82
|
-
# 500-line prompt that breaks when models update
|
|
83
|
-
|
|
84
|
-
# How do I know that there isn't an even better prompt? (you don't)
|
|
85
|
-
# -> proving 'best possible performance' is impossible
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
**🧪 Testing Nightmares**
|
|
89
|
-
```python
|
|
90
|
-
# How do you unit test this?
|
|
91
|
-
result = llm.invoke(prompt) # Hope for valid JSON
|
|
92
|
-
data = json.loads(result.content) # Crashes in production
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
**📐 Rigid topology and tight coupling**
|
|
96
|
-
```python
|
|
97
|
-
# Want to add a new agent? Rewrite the entire graph.
|
|
98
|
-
workflow.add_edge("agent_a", "agent_b")
|
|
99
|
-
workflow.add_edge("agent_b", "agent_c")
|
|
100
|
-
# Add agent_d? Start rewiring...
|
|
101
|
-
```
|
|
102
|
-
|
|
103
|
-
**💀 Single point of failure: Orchestrator dies? Everything dies.**
|
|
104
|
-
```python
|
|
105
|
-
# Orchestrator dies? Everything dies.
|
|
106
|
-
```
|
|
107
|
-
|
|
108
|
-
**🧠 God object anti-pattern:**
|
|
109
|
-
```python
|
|
110
|
-
# One orchestrator needs domain knowledge of 20+ agents to route correctly
|
|
111
|
-
# Orchestrator 'guesses' next agent based on a natural language description.
|
|
112
|
-
# Not suitable for critical systems.
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
These aren't framework limitations, they're **architectural choices** that don't scale.
|
|
116
|
-
|
|
117
|
-
These challenges are solvable—decades of experience with microservices have taught us hard lessons about decoupling, orchestration, and reliability. Let's apply those lessons!
|
|
118
|
-
|
|
119
|
-
---
|
|
120
|
-
|
|
121
|
-
## The Flock Approach
|
|
122
|
-
|
|
123
|
-
Flock takes a different path, combining two proven patterns:
|
|
124
|
-
|
|
125
|
-
### 1. Declarative Type Contracts (Not Prompts)
|
|
126
|
-
|
|
127
|
-
**Traditional approach:**
|
|
128
|
-
```python
|
|
129
|
-
prompt = """You are an expert software engineer and bug analyst. Your task is to analyze bug reports and provide structured diagnostic information.
|
|
130
|
-
|
|
131
|
-
INSTRUCTIONS:
|
|
132
|
-
1. Read the bug report carefully
|
|
133
|
-
2. Determine the severity level (must be exactly one of: Critical, High, Medium, Low)
|
|
134
|
-
3. Classify the bug category (e.g., "performance", "security", "UI", "data corruption")
|
|
135
|
-
4. Formulate a root cause hypothesis (minimum 50 characters)
|
|
136
|
-
5. Assign a confidence score between 0.0 and 1.0
|
|
137
|
-
|
|
138
|
-
OUTPUT FORMAT:
|
|
139
|
-
You MUST return valid JSON with this exact structure:
|
|
140
|
-
{
|
|
141
|
-
"severity": "string (Critical|High|Medium|Low)",
|
|
142
|
-
"category": "string",
|
|
143
|
-
"root_cause_hypothesis": "string (minimum 50 characters)",
|
|
144
|
-
"confidence_score": "number (0.0 to 1.0)"
|
|
145
|
-
}
|
|
146
|
-
|
|
147
|
-
VALIDATION RULES:
|
|
148
|
-
- severity: Must be exactly one of: Critical, High, Medium, Low (case-sensitive)
|
|
149
|
-
- category: Must be a single word or short phrase describing the bug type
|
|
150
|
-
- root_cause_hypothesis: Must be at least 50 characters long and explain the likely cause
|
|
151
|
-
- confidence_score: Must be a decimal number between 0.0 and 1.0 inclusive
|
|
152
|
-
|
|
153
|
-
EXAMPLES:
|
|
154
|
-
Input: "App crashes when user clicks submit button"
|
|
155
|
-
Output: {"severity": "Critical", "category": "crash", "root_cause_hypothesis": "Null pointer exception in form validation logic when required fields are empty", "confidence_score": 0.85}
|
|
156
|
-
|
|
157
|
-
Input: "Login button has wrong color"
|
|
158
|
-
Output: {"severity": "Low", "category": "UI", "root_cause_hypothesis": "CSS class override not applied correctly in the theme configuration", "confidence_score": 0.9}
|
|
159
|
-
|
|
160
|
-
IMPORTANT:
|
|
161
|
-
- Do NOT include any explanatory text before or after the JSON
|
|
162
|
-
- Do NOT use markdown code blocks (no ```json```)
|
|
163
|
-
- Do NOT add comments in the JSON
|
|
164
|
-
- Ensure the JSON is valid and parseable
|
|
165
|
-
- If you cannot determine something, use your best judgment
|
|
166
|
-
- Never return null values
|
|
167
|
-
|
|
168
|
-
Now analyze this bug report:
|
|
169
|
-
{bug_report_text}"""
|
|
170
|
-
|
|
171
|
-
result = llm.invoke(prompt) # 500-line prompt that breaks when models update
|
|
172
|
-
# Then parse and hope it's valid JSON
|
|
173
|
-
data = json.loads(result.content) # Crashes in production 🔥
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
**The Flock way:**
|
|
177
|
-
```python
|
|
178
|
-
@flock_type
|
|
179
|
-
class BugDiagnosis(BaseModel):
|
|
180
|
-
severity: str = Field(pattern="^(Critical|High|Medium|Low)$")
|
|
181
|
-
category: str = Field(description="Bug category")
|
|
182
|
-
root_cause_hypothesis: str = Field(min_length=50)
|
|
183
|
-
confidence_score: float = Field(ge=0.0, le=1.0)
|
|
184
|
-
|
|
185
|
-
# The schema IS the instruction. No 500-line prompt needed.
|
|
186
|
-
agent.consumes(BugReport).publishes(BugDiagnosis)
|
|
187
|
-
```
|
|
188
|
-
|
|
189
|
-
<p align="center">
|
|
190
|
-
<img alt="Flock Banner" src="docs/assets/images/bug_diagnosis.png" width="1000">
|
|
191
|
-
</p>
|
|
192
|
-
|
|
193
|
-
**Why this matters:**
|
|
194
|
-
- ✅ **Survives model upgrades** - GPT-6 will still understand Pydantic schemas
|
|
195
|
-
- ✅ **Runtime validation** - Errors caught at parse time, not in production
|
|
196
|
-
- ✅ **Testable** - Mock inputs/outputs with concrete types
|
|
197
|
-
- ✅ **Self-documenting** - The code tells you what agents do
|
|
198
|
-
|
|
199
|
-
### 2. Blackboard Architecture (Not Directed Graphs)
|
|
200
|
-
|
|
201
|
-
**Graph-based approach:**
|
|
202
|
-
```python
|
|
203
|
-
# Explicit workflow with hardcoded edges
|
|
204
|
-
workflow.add_edge("radiologist", "diagnostician")
|
|
205
|
-
workflow.add_edge("lab_tech", "diagnostician")
|
|
206
|
-
# Add performance_analyzer? Rewrite the graph.
|
|
207
|
-
```
|
|
208
|
-
|
|
209
|
-
**The Flock way (blackboard):**
|
|
210
|
-
```python
|
|
211
|
-
# Agents subscribe to types, workflows emerge
|
|
212
|
-
radiologist = flock.agent("radiologist").consumes(Scan).publishes(XRayAnalysis)
|
|
213
|
-
lab_tech = flock.agent("lab_tech").consumes(Scan).publishes(LabResults)
|
|
214
|
-
diagnostician = flock.agent("diagnostician").consumes(XRayAnalysis, LabResults).publishes(Diagnosis)
|
|
215
|
-
|
|
216
|
-
# Add performance_analyzer? Just subscribe it:
|
|
217
|
-
performance = flock.agent("perf").consumes(Scan).publishes(PerfAnalysis)
|
|
218
|
-
# Done. No graph rewiring. Diagnostician can optionally consume it.
|
|
219
|
-
```
|
|
220
|
-
|
|
221
|
-
**What just happened:**
|
|
222
|
-
- ✅ **Parallel execution** - Radiologist and lab_tech run concurrently (automatic)
|
|
223
|
-
- ✅ **Dependency resolution** - Diagnostician waits for both inputs (automatic)
|
|
224
|
-
- ✅ **Loose coupling** - Agents don't know about each other, just data types
|
|
225
|
-
- ✅ **Scalable** - O(n) complexity, not O(n²) edges
|
|
226
|
-
|
|
227
|
-
**This is not a new idea.** Blackboard architecture has powered groundbreaking AI systems since the 1970s (Hearsay-II, HASP/SIAP, BB1). We're applying proven patterns to modern LLMs.
|
|
228
|
-
|
|
229
|
-
---
|
|
230
|
-
|
|
231
|
-
## Quick Start (60 Seconds)
|
|
232
|
-
|
|
233
|
-
```bash
|
|
234
|
-
pip install flock-core
|
|
235
|
-
export OPENAI_API_KEY="sk-..."
|
|
236
|
-
# Optional: export DEFAULT_MODEL (falls back to hard-coded default if unset)
|
|
237
|
-
export DEFAULT_MODEL="openai/gpt-4.1"
|
|
238
|
-
```
|
|
239
|
-
|
|
240
|
-
```python
|
|
241
|
-
import os
|
|
242
|
-
import asyncio
|
|
243
|
-
from pydantic import BaseModel, Field
|
|
244
|
-
from flock import Flock, flock_type
|
|
245
|
-
|
|
246
|
-
# 1. Define typed artifacts
|
|
247
|
-
@flock_type
|
|
248
|
-
class CodeSubmission(BaseModel):
|
|
249
|
-
code: str
|
|
250
|
-
language: str
|
|
251
|
-
|
|
252
|
-
@flock_type
|
|
253
|
-
class BugAnalysis(BaseModel):
|
|
254
|
-
bugs_found: list[str]
|
|
255
|
-
severity: str = Field(pattern="^(Critical|High|Medium|Low|None)$")
|
|
256
|
-
confidence: float = Field(ge=0.0, le=1.0)
|
|
257
|
-
|
|
258
|
-
@flock_type
|
|
259
|
-
class SecurityAnalysis(BaseModel):
|
|
260
|
-
vulnerabilities: list[str]
|
|
261
|
-
risk_level: str = Field(pattern="^(Critical|High|Medium|Low|None)$")
|
|
262
|
-
|
|
263
|
-
@flock_type
|
|
264
|
-
class FinalReview(BaseModel):
|
|
265
|
-
overall_assessment: str = Field(pattern="^(Approve|Approve with Changes|Reject)$")
|
|
266
|
-
action_items: list[str]
|
|
267
|
-
|
|
268
|
-
# 2. Create the blackboard
|
|
269
|
-
flock = Flock(os.getenv("DEFAULT_MODEL", "openai/gpt-4.1"))
|
|
270
|
-
|
|
271
|
-
# 3. Agents subscribe to types (NO graph wiring!)
|
|
272
|
-
bug_detector = flock.agent("bug_detector").consumes(CodeSubmission).publishes(BugAnalysis)
|
|
273
|
-
security_auditor = flock.agent("security_auditor").consumes(CodeSubmission).publishes(SecurityAnalysis)
|
|
274
|
-
|
|
275
|
-
# AND gate: This agent AUTOMATICALLY waits for BOTH analyses before triggering
|
|
276
|
-
final_reviewer = flock.agent("final_reviewer").consumes(BugAnalysis, SecurityAnalysis).publishes(FinalReview)
|
|
277
|
-
|
|
278
|
-
# 4. Run with real-time dashboard
|
|
279
|
-
async def main():
|
|
280
|
-
await flock.serve(dashboard=True)
|
|
281
|
-
|
|
282
|
-
asyncio.run(main())
|
|
283
|
-
```
|
|
284
|
-
|
|
285
|
-
**What happened:**
|
|
286
|
-
- Bug detector and security auditor ran **in parallel** (both consume CodeSubmission)
|
|
287
|
-
- Final reviewer **automatically waited** for both
|
|
288
|
-
- **Zero prompts written** - types defined the behavior
|
|
289
|
-
- **Zero graph edges** - subscriptions created the workflow
|
|
290
|
-
- **Full type safety** - Pydantic validates all outputs
|
|
291
|
-
|
|
292
|
-
---
|
|
293
|
-
|
|
294
|
-
## Core Concepts
|
|
295
|
-
|
|
296
|
-
### Typed Artifacts (The Vocabulary)
|
|
297
|
-
|
|
298
|
-
Every piece of data on the blackboard is a validated Pydantic model:
|
|
299
|
-
|
|
300
|
-
```python
|
|
301
|
-
@flock_type
|
|
302
|
-
class PatientDiagnosis(BaseModel):
|
|
303
|
-
condition: str = Field(min_length=10)
|
|
304
|
-
confidence: float = Field(ge=0.0, le=1.0)
|
|
305
|
-
recommended_treatment: list[str] = Field(min_length=1)
|
|
306
|
-
follow_up_required: bool
|
|
307
|
-
```
|
|
308
|
-
|
|
309
|
-
**Benefits:**
|
|
310
|
-
- Runtime validation ensures quality
|
|
311
|
-
- Field constraints prevent bad outputs
|
|
312
|
-
- Self-documenting data structures
|
|
313
|
-
- Version-safe (types survive model updates)
|
|
314
|
-
|
|
315
|
-
### Agent Subscriptions (The Rules)
|
|
316
|
-
|
|
317
|
-
Agents declare what they consume and produce:
|
|
318
|
-
|
|
319
|
-
```python
|
|
320
|
-
analyzer = (
|
|
321
|
-
flock.agent("analyzer")
|
|
322
|
-
.description("Analyzes patient scans") # Optional: improves multi-agent coordination
|
|
323
|
-
.consumes(PatientScan) # What triggers this agent
|
|
324
|
-
.publishes(PatientDiagnosis) # What it produces
|
|
325
|
-
)
|
|
326
|
-
```
|
|
327
|
-
|
|
328
|
-
**Logic Operations (AND/OR Gates):**
|
|
329
|
-
|
|
330
|
-
Flock provides intuitive syntax for coordinating multiple input types:
|
|
331
|
-
|
|
332
|
-
```python
|
|
333
|
-
# AND gate: Wait for BOTH types before triggering
|
|
334
|
-
diagnostician = flock.agent("diagnostician").consumes(XRayAnalysis, LabResults).publishes(Diagnosis)
|
|
335
|
-
# Agent triggers only when both XRayAnalysis AND LabResults are available
|
|
336
|
-
|
|
337
|
-
# OR gate: Trigger on EITHER type (via chaining)
|
|
338
|
-
alert_handler = flock.agent("alerts").consumes(SystemAlert).consumes(UserAlert).publishes(Response)
|
|
339
|
-
# Agent triggers when SystemAlert OR UserAlert is published
|
|
340
|
-
|
|
341
|
-
# Count-based AND gate: Wait for MULTIPLE instances of the same type
|
|
342
|
-
aggregator = flock.agent("aggregator").consumes(Order, Order, Order).publishes(BatchSummary)
|
|
343
|
-
# Agent triggers when THREE Order artifacts are available
|
|
344
|
-
|
|
345
|
-
# Mixed counts: Different requirements per type
|
|
346
|
-
validator = flock.agent("validator").consumes(Image, Image, Metadata).publishes(ValidationResult)
|
|
347
|
-
# Agent triggers when TWO Images AND ONE Metadata are available
|
|
348
|
-
```
|
|
349
|
-
|
|
350
|
-
**What just happened:**
|
|
351
|
-
- ✅ **Natural syntax** - Code clearly expresses intent ("wait for 3 orders")
|
|
352
|
-
- ✅ **Order-independent** - Artifacts can arrive in any sequence
|
|
353
|
-
- ✅ **Latest wins** - If 4 As arrive but need 3, uses the 3 most recent
|
|
354
|
-
- ✅ **Zero configuration** - No manual coordination logic needed
|
|
355
|
-
|
|
356
|
-
**Advanced subscriptions unlock crazy powerful patterns:**
|
|
357
|
-
|
|
358
|
-
<p align="center">
|
|
359
|
-
<img alt="Event Join" src="docs/assets/images/join.png" width="800">
|
|
360
|
-
</p>
|
|
361
|
-
|
|
362
|
-
```python
|
|
363
|
-
# 🎯 Predicates - Smart filtering (only process critical cases)
|
|
364
|
-
urgent_care = flock.agent("urgent").consumes(
|
|
365
|
-
Diagnosis,
|
|
366
|
-
where=lambda d: d.severity in ["Critical", "High"] # Conditional routing!
|
|
367
|
-
)
|
|
368
|
-
|
|
369
|
-
# 📦 BatchSpec - Cost optimization (process 10 at once = 90% cheaper API calls)
|
|
370
|
-
payment_processor = flock.agent("payments").consumes(
|
|
371
|
-
Transaction,
|
|
372
|
-
batch=BatchSpec(size=25, timeout=timedelta(seconds=30)) # $5 saved per batch!
|
|
373
|
-
)
|
|
374
|
-
|
|
375
|
-
# 🔗 JoinSpec - Data correlation (match orders + shipments by ID)
|
|
376
|
-
customer_service = flock.agent("notifications").consumes(
|
|
377
|
-
Order,
|
|
378
|
-
Shipment,
|
|
379
|
-
join=JoinSpec(by=lambda x: x.order_id, within=timedelta(hours=24)) # Correlated!
|
|
380
|
-
)
|
|
381
|
-
|
|
382
|
-
# 🏭 Combined Features - Correlate sensors, THEN batch for analysis
|
|
383
|
-
quality_control = flock.agent("qc").consumes(
|
|
384
|
-
TemperatureSensor,
|
|
385
|
-
PressureSensor,
|
|
386
|
-
join=JoinSpec(by=lambda x: x.device_id, within=timedelta(seconds=30)),
|
|
387
|
-
batch=BatchSpec(size=5, timeout=timedelta(seconds=45)) # IoT at scale!
|
|
388
|
-
)
|
|
389
|
-
```
|
|
390
|
-
|
|
391
|
-
**What just happened:**
|
|
392
|
-
- ✅ **Predicates** route work by business rules ("only critical severity")
|
|
393
|
-
- ✅ **BatchSpec** optimizes costs (25 transactions = 1 API call instead of 25)
|
|
394
|
-
- ✅ **JoinSpec** correlates related data (orders ↔ shipments, sensors ↔ readings)
|
|
395
|
-
- ✅ **Combined** delivers production-grade multi-stage pipelines
|
|
396
|
-
|
|
397
|
-
**Real-world impact:**
|
|
398
|
-
- 💰 E-commerce: Save $5 per batch on payment processing fees
|
|
399
|
-
- 🏥 Healthcare: Correlate patient scans + lab results for diagnosis
|
|
400
|
-
- 🏭 Manufacturing: Monitor 1000+ IoT sensors with efficient batching
|
|
401
|
-
- 📊 Finance: Match trades + confirmations within 5-minute windows
|
|
402
|
-
|
|
403
|
-
<p align="center">
|
|
404
|
-
<img alt="Event Batch" src="docs/assets/images/batch.png" width="800">
|
|
405
|
-
</p>
|
|
406
|
-
|
|
407
|
-
### 🌟 Fan-Out Publishing (New in 0.5)
|
|
408
|
-
|
|
409
|
-
**Produce multiple outputs from a single agent execution:**
|
|
410
|
-
|
|
411
|
-
```python
|
|
412
|
-
# Generate 10 diverse product ideas from one brief
|
|
413
|
-
idea_generator = (
|
|
414
|
-
flock.agent("generator")
|
|
415
|
-
.consumes(ProductBrief)
|
|
416
|
-
.publishes(ProductIdea, fan_out=10) # Produces 10 ideas per brief!
|
|
417
|
-
)
|
|
418
|
-
|
|
419
|
-
# With WHERE filtering - only publish high-quality ideas
|
|
420
|
-
idea_generator = (
|
|
421
|
-
flock.agent("generator")
|
|
422
|
-
.consumes(ProductBrief)
|
|
423
|
-
.publishes(
|
|
424
|
-
ProductIdea,
|
|
425
|
-
fan_out=20, # Generate 20 candidates
|
|
426
|
-
where=lambda idea: idea.score >= 8.0 # Only publish score >= 8
|
|
427
|
-
)
|
|
428
|
-
)
|
|
429
|
-
|
|
430
|
-
# With VALIDATE - enforce quality standards
|
|
431
|
-
code_reviewer = (
|
|
432
|
-
flock.agent("reviewer")
|
|
433
|
-
.consumes(CodeSubmission)
|
|
434
|
-
.publishes(
|
|
435
|
-
BugReport,
|
|
436
|
-
fan_out=5,
|
|
437
|
-
validate=lambda bug: bug.severity in ["Critical", "High", "Medium", "Low"]
|
|
438
|
-
)
|
|
439
|
-
)
|
|
440
|
-
|
|
441
|
-
# With Dynamic Visibility - control access per artifact
|
|
442
|
-
notification_agent = (
|
|
443
|
-
flock.agent("notifier")
|
|
444
|
-
.consumes(Alert)
|
|
445
|
-
.publishes(
|
|
446
|
-
Notification,
|
|
447
|
-
fan_out=3,
|
|
448
|
-
visibility=lambda n: PrivateVisibility(agents=[n.recipient]) # Dynamic!
|
|
449
|
-
)
|
|
450
|
-
)
|
|
451
|
-
```
|
|
452
|
-
|
|
453
|
-
**What just happened:**
|
|
454
|
-
- ✅ **fan_out=N** - Agent produces N artifacts per execution (not just 1!)
|
|
455
|
-
- ✅ **where** - Filter outputs before publishing (reduce noise, save costs)
|
|
456
|
-
- ✅ **validate** - Enforce quality standards (fail-fast on bad outputs)
|
|
457
|
-
- ✅ **Dynamic visibility** - Control access per artifact based on content
|
|
458
|
-
|
|
459
|
-
**Real-world impact:**
|
|
460
|
-
- 🎯 **Content Generation** - Generate 10 blog post ideas, filter to top 3 by score
|
|
461
|
-
- 🐛 **Code Review** - Produce 5 potential bugs, validate severity levels
|
|
462
|
-
- 📧 **Notifications** - Create 3 notification variants, target specific agents
|
|
463
|
-
- 🧪 **A/B Testing** - Generate N variations, filter by quality metrics
|
|
464
|
-
|
|
465
|
-
**🤯 Multi-Output Fan-Out (New in 0.5)**
|
|
466
|
-
|
|
467
|
-
**The truly mind-blowing part:** Fan-out works across **multiple output types**:
|
|
468
|
-
|
|
469
|
-
```python
|
|
470
|
-
# Generate 3 of EACH type = 9 total artifacts in ONE LLM call!
|
|
471
|
-
multi_master = (
|
|
472
|
-
flock.agent("multi_master")
|
|
473
|
-
.consumes(Idea)
|
|
474
|
-
.publishes(Movie, MovieScript, MovieCampaign, fan_out=3)
|
|
475
|
-
)
|
|
476
|
-
|
|
477
|
-
# Single execution produces:
|
|
478
|
-
# - 3 complete Movie artifacts (with title, genre, cast, plot)
|
|
479
|
-
# - 3 complete MovieScript artifacts (with characters, scenes, pages)
|
|
480
|
-
# - 3 complete MovieCampaign artifacts (with taglines, poster descriptions)
|
|
481
|
-
# = 9 complex artifacts, ~100+ fields total, full Pydantic validation, ONE LLM call!
|
|
482
|
-
|
|
483
|
-
await flock.publish(Idea(story_idea="An action thriller set in space"))
|
|
484
|
-
await flock.run_until_idle()
|
|
485
|
-
|
|
486
|
-
# Result: 9 artifacts on the blackboard, all validated, all ready
|
|
487
|
-
movies = await flock.store.get_by_type(Movie) # 3 movies
|
|
488
|
-
scripts = await flock.store.get_by_type(MovieScript) # 3 scripts
|
|
489
|
-
campaigns = await flock.store.get_by_type(MovieCampaign) # 3 campaigns
|
|
490
|
-
```
|
|
491
|
-
|
|
492
|
-
**Why this is revolutionary:**
|
|
493
|
-
- ⚡ **Massive efficiency** - 1 LLM call generates 9 production-ready artifacts
|
|
494
|
-
- ✅ **Full validation** - All 100+ fields validated with Pydantic constraints
|
|
495
|
-
- 🎯 **Coherent generation** - Movie/Script/Campaign are thematically aligned (same LLM context)
|
|
496
|
-
- 💰 **Cost optimized** - 9 artifacts for the price of 1 API call
|
|
497
|
-
|
|
498
|
-
**Can any other agent framework do this?** We haven't found one. 🚀
|
|
499
|
-
|
|
500
|
-
**📖 [Full Fan-Out Guide →](https://whiteducksoftware.github.io/flock/guides/fan-out/)**
|
|
501
|
-
|
|
502
|
-
### Visibility Controls (The Security)
|
|
503
|
-
|
|
504
|
-
**Unlike other frameworks, Flock has zero-trust security built-in:**
|
|
505
|
-
|
|
506
|
-
```python
|
|
507
|
-
# Multi-tenancy (SaaS isolation)
|
|
508
|
-
agent.publishes(CustomerData, visibility=TenantVisibility(tenant_id="customer_123"))
|
|
509
|
-
|
|
510
|
-
# Explicit allowlist (HIPAA compliance)
|
|
511
|
-
agent.publishes(MedicalRecord, visibility=PrivateVisibility(agents={"physician", "nurse"}))
|
|
512
|
-
|
|
513
|
-
# Role-based access control
|
|
514
|
-
agent.identity(AgentIdentity(name="analyst", labels={"clearance:secret"}))
|
|
515
|
-
agent.publishes(IntelReport, visibility=LabelledVisibility(required_labels={"clearance:secret"}))
|
|
516
|
-
|
|
517
|
-
# Time-delayed release (embargo periods)
|
|
518
|
-
artifact.visibility = AfterVisibility(ttl=timedelta(hours=24), then=PublicVisibility())
|
|
519
|
-
|
|
520
|
-
# Public (default)
|
|
521
|
-
agent.publishes(PublicReport, visibility=PublicVisibility())
|
|
522
|
-
```
|
|
523
|
-
|
|
524
|
-
**Visibility has a dual purpose:** It controls both which agents can be **triggered** by an artifact AND which artifacts agents can **see** in their context. This ensures consistent security across agent execution and data access—agents cannot bypass visibility controls through subscription filters or context providers.
|
|
525
|
-
|
|
526
|
-
**Why this matters:** Financial services, healthcare, defense, SaaS platforms all need this for compliance. Other frameworks make you build it yourself.
|
|
527
|
-
|
|
528
|
-
---
|
|
529
|
-
|
|
530
|
-
### 🔒 Architecturally Impossible to Bypass Security
|
|
531
|
-
|
|
532
|
-
**Here's what makes Flock different:** In most frameworks, security is something you remember to add. In Flock, **it's architecturally impossible to forget.**
|
|
533
|
-
|
|
534
|
-
Every context provider in Flock inherits from `BaseContextProvider`, which enforces visibility filtering **automatically**. You literally cannot create a provider that forgets to check permissions—the security logic is baked into the base class and executes before your custom code even runs.
|
|
535
|
-
|
|
536
|
-
**What this means in practice:**
|
|
537
|
-
|
|
538
|
-
```python
|
|
539
|
-
# ❌ Other frameworks: Security is your responsibility (easy to forget!)
|
|
540
|
-
class MyProvider:
|
|
541
|
-
async def get_context(self, agent):
|
|
542
|
-
artifacts = store.get_all() # OOPS! Forgot to check visibility!
|
|
543
|
-
return artifacts # 🔥 Security vulnerability
|
|
544
|
-
|
|
545
|
-
# ✅ Flock: Security is enforced automatically (impossible to bypass!)
|
|
546
|
-
class MyProvider(BaseContextProvider):
|
|
547
|
-
async def get_artifacts(self, request):
|
|
548
|
-
artifacts = await store.query_artifacts(...)
|
|
549
|
-
return artifacts # ✨ Visibility filtering happens automatically!
|
|
550
|
-
# BaseContextProvider calls .visibility.allows() for you
|
|
551
|
-
# You CANNOT bypass this - it's enforced by the architecture
|
|
552
|
-
```
|
|
553
|
-
|
|
554
|
-
**Built-in providers (all inherit BaseContextProvider):**
|
|
555
|
-
- `DefaultContextProvider` - Full blackboard access (visibility-filtered)
|
|
556
|
-
- `CorrelatedContextProvider` - Workflow isolation (visibility-filtered)
|
|
557
|
-
- `RecentContextProvider` - Token cost control (visibility-filtered)
|
|
558
|
-
- `TimeWindowContextProvider` - Time-based filtering (visibility-filtered)
|
|
559
|
-
- `EmptyContextProvider` - Stateless agents (zero context)
|
|
560
|
-
- `FilteredContextProvider` - Custom filtering (visibility-filtered)
|
|
561
|
-
|
|
562
|
-
**Every single one enforces visibility automatically. Zero chance of accidentally leaking data.**
|
|
563
|
-
|
|
564
|
-
This isn't just convenient—it's **security by design**. When you're building HIPAA-compliant healthcare systems or SOC2-certified SaaS platforms, "impossible to bypass even by accident" is the only acceptable standard.
|
|
565
|
-
|
|
566
|
-
---
|
|
567
|
-
|
|
568
|
-
### Context Providers (The Smart Filter)
|
|
569
|
-
|
|
570
|
-
**Control what agents see with custom Context Providers:**
|
|
571
|
-
|
|
572
|
-
```python
|
|
573
|
-
from flock.context_provider import FilteredContextProvider, PasswordRedactorProvider
|
|
574
|
-
from flock.store import FilterConfig
|
|
575
|
-
|
|
576
|
-
# Global filtering - all agents see only urgent items
|
|
577
|
-
flock = Flock(
|
|
578
|
-
"openai/gpt-4.1",
|
|
579
|
-
context_provider=FilteredContextProvider(FilterConfig(tags={"urgent"}))
|
|
580
|
-
)
|
|
581
|
-
|
|
582
|
-
# Per-agent overrides - specialized context per agent
|
|
583
|
-
error_agent = flock.agent("errors").consumes(Log).publishes(Alert)
|
|
584
|
-
error_agent.context_provider = FilteredContextProvider(FilterConfig(tags={"ERROR"}))
|
|
585
|
-
|
|
586
|
-
# Production-ready password filtering
|
|
587
|
-
from examples.context_provider import PasswordRedactorProvider
|
|
588
|
-
flock = Flock(
|
|
589
|
-
"openai/gpt-4.1",
|
|
590
|
-
context_provider=PasswordRedactorProvider() # Auto-redacts sensitive data!
|
|
591
|
-
)
|
|
592
|
-
```
|
|
593
|
-
|
|
594
|
-
**What just happened:**
|
|
595
|
-
- ✅ **Filtered context** - Agents see only relevant artifacts (save tokens, improve performance)
|
|
596
|
-
- ✅ **Security boundary** - Visibility enforcement + custom filtering (mandatory, cannot bypass)
|
|
597
|
-
- ✅ **Sensitive data protection** - Auto-redact passwords, API keys, credit cards, SSN, JWT tokens
|
|
598
|
-
- ✅ **Per-agent specialization** - Different agents, different context rules
|
|
599
|
-
|
|
600
|
-
**Production patterns:**
|
|
601
|
-
```python
|
|
602
|
-
# Password/secret redaction (copy-paste ready!)
|
|
603
|
-
provider = PasswordRedactorProvider(
|
|
604
|
-
custom_patterns={"internal_id": r"ID-\d{6}"},
|
|
605
|
-
redaction_text="[REDACTED]"
|
|
606
|
-
)
|
|
607
|
-
|
|
608
|
-
# Role-based access control
|
|
609
|
-
junior_agent.context_provider = FilteredContextProvider(FilterConfig(tags={"ERROR"}))
|
|
610
|
-
senior_agent.context_provider = FilteredContextProvider(FilterConfig(tags={"ERROR", "WARN"}))
|
|
611
|
-
admin_agent.context_provider = None # See everything (uses default)
|
|
612
|
-
|
|
613
|
-
# Multi-tenant isolation
|
|
614
|
-
agent.context_provider = FilteredContextProvider(
|
|
615
|
-
FilterConfig(tags={"tenant:customer_123"})
|
|
616
|
-
)
|
|
617
|
-
```
|
|
618
|
-
|
|
619
|
-
**Why this matters:** Reduce token costs (90%+ with smart filtering), protect sensitive data (auto-redact secrets), improve performance (agents see only what they need).
|
|
620
|
-
|
|
621
|
-
**📖 [Learn more: Context Providers Guide](https://whiteducksoftware.github.io/flock/guides/context-providers/) | [Steal production code →](examples/08-context-provider/)**
|
|
622
|
-
|
|
623
|
-
### Persistent Blackboard History
|
|
624
|
-
|
|
625
|
-
The in-memory store is great for local development, but production teams need durability. The `SQLiteBlackboardStore` turns the blackboard into a persistent event log with first-class ergonomics:
|
|
626
|
-
|
|
627
|
-
**What you get:**
|
|
628
|
-
- **Long-lived artifacts** — Every field (payload, tags, partition keys, visibility) stored for replay, audits, and postmortems
|
|
629
|
-
- **Historical APIs** — `/api/v1/artifacts`, `/summary`, and `/agents/{agent_id}/history-summary` expose pagination, filtering, and consumption counts
|
|
630
|
-
- **Dashboard integration** — The **Historical Blackboard** view preloads persisted history, enriches the graph with consumer metadata, and highlights retention windows
|
|
631
|
-
- **Operational tooling** — CLI helpers (`init-sqlite-store`, `sqlite-maintenance --delete-before ... --vacuum`) make schema setup and retention policies scriptable
|
|
632
|
-
|
|
633
|
-
**Quick start:**
|
|
634
|
-
```python
|
|
635
|
-
from flock import Flock
|
|
636
|
-
from flock.store import SQLiteBlackboardStore
|
|
637
|
-
|
|
638
|
-
store = SQLiteBlackboardStore(".flock/blackboard.db")
|
|
639
|
-
await store.ensure_schema()
|
|
640
|
-
flock = Flock("openai/gpt-4.1", store=store)
|
|
641
|
-
```
|
|
642
|
-
|
|
643
|
-
**Try it:** Run `examples/02-the-blackboard/01_persistent_pizza.py` to generate history, then launch `examples/03-the-dashboard/04_persistent_pizza_dashboard.py` to explore previous runs, consumption trails, and retention banners.
|
|
644
|
-
|
|
645
|
-
### Batching Pattern: Parallel Execution Control
|
|
646
|
-
|
|
647
|
-
**A key differentiator:** The separation of `publish()` and `run_until_idle()` enables parallel execution.
|
|
648
|
-
|
|
649
|
-
```python
|
|
650
|
-
# ✅ EFFICIENT: Batch publish, then run in parallel
|
|
651
|
-
for review in customer_reviews:
|
|
652
|
-
await flock.publish(review) # Just scheduling work
|
|
653
|
-
|
|
654
|
-
await flock.run_until_idle() # All sentiment_analyzer agents run concurrently!
|
|
655
|
-
|
|
656
|
-
# Get all results
|
|
657
|
-
analyses = await flock.store.get_by_type(SentimentAnalysis)
|
|
658
|
-
# 100 analyses completed in ~1x single review processing time!
|
|
659
|
-
```
|
|
660
|
-
|
|
661
|
-
**Why this separation matters:**
|
|
662
|
-
- ⚡ **Parallel execution** - Process 100 customer reviews concurrently
|
|
663
|
-
- 🎯 **Batch control** - Publish multiple artifacts, execute once
|
|
664
|
-
- 🔄 **Multi-type workflows** - Publish different types, trigger different agents in parallel
|
|
665
|
-
- 📊 **Better performance** - Process 1000 items in the time it takes to process 1
|
|
666
|
-
|
|
667
|
-
**Comparison to other patterns:**
|
|
668
|
-
```python
|
|
669
|
-
# ❌ If run_until_idle() was automatic (like most frameworks):
|
|
670
|
-
for review in customer_reviews:
|
|
671
|
-
await flock.publish(review) # Would wait for completion each time!
|
|
672
|
-
# Total time: 100x single execution (sequential)
|
|
673
|
-
|
|
674
|
-
# ✅ With explicit batching:
|
|
675
|
-
for review in customer_reviews:
|
|
676
|
-
await flock.publish(review) # Fast: just queuing
|
|
677
|
-
await flock.run_until_idle()
|
|
678
|
-
# Total time: ~1x single execution (parallel)
|
|
679
|
-
```
|
|
680
|
-
|
|
681
|
-
### Agent Components (Agent Lifecycle Hooks)
|
|
682
|
-
|
|
683
|
-
**Extend agent behavior through composable lifecycle hooks:**
|
|
684
|
-
|
|
685
|
-
Agent components let you inject custom logic at specific points in an agent's execution without modifying core agent code:
|
|
686
|
-
|
|
687
|
-
```python
|
|
688
|
-
from flock.components import AgentComponent
|
|
689
|
-
|
|
690
|
-
# Custom component: Log inputs/outputs
|
|
691
|
-
class LoggingComponent(AgentComponent):
|
|
692
|
-
async def on_pre_evaluate(self, agent, ctx, inputs):
|
|
693
|
-
logger.info(f"Agent {agent.name} evaluating: {inputs}")
|
|
694
|
-
return inputs # Pass through unchanged
|
|
695
|
-
|
|
696
|
-
async def on_post_evaluate(self, agent, ctx, inputs, result):
|
|
697
|
-
logger.info(f"Agent {agent.name} produced: {result}")
|
|
698
|
-
return result
|
|
699
|
-
|
|
700
|
-
# Attach to any agent
|
|
701
|
-
analyzer = (
|
|
702
|
-
flock.agent("analyzer")
|
|
703
|
-
.consumes(Data)
|
|
704
|
-
.publishes(Report)
|
|
705
|
-
.with_utilities(LoggingComponent())
|
|
706
|
-
)
|
|
707
|
-
```
|
|
708
|
-
|
|
709
|
-
**Built-in components**: Rate limiting, caching, metrics collection, budget tracking, guardrails
|
|
710
|
-
|
|
711
|
-
**📖 [Learn more: Agent Components Guide](https://whiteducksoftware.github.io/flock/guides/components/)**
|
|
712
|
-
|
|
713
|
-
---
|
|
714
|
-
|
|
715
|
-
### Orchestrator Components (Orchestrator Lifecycle Hooks)
|
|
716
|
-
|
|
717
|
-
**Extend orchestrator behavior through composable lifecycle hooks:**
|
|
718
|
-
|
|
719
|
-
Orchestrator components let you inject custom logic into the orchestrator's scheduling pipeline:
|
|
720
|
-
|
|
721
|
-
```python
|
|
722
|
-
from flock.orchestrator_component import OrchestratorComponent, ScheduleDecision
|
|
723
|
-
|
|
724
|
-
# Custom component: Skip scheduling during maintenance window
|
|
725
|
-
class MaintenanceWindowComponent(OrchestratorComponent):
|
|
726
|
-
async def on_before_schedule(self, orch, artifact, agent, subscription):
|
|
727
|
-
if self.is_maintenance_window():
|
|
728
|
-
logger.info(f"Deferring {agent.name} during maintenance")
|
|
729
|
-
return ScheduleDecision.DEFER
|
|
730
|
-
return ScheduleDecision.CONTINUE
|
|
731
|
-
|
|
732
|
-
# Add to orchestrator
|
|
733
|
-
flock = Flock("openai/gpt-4.1")
|
|
734
|
-
flock.add_component(MaintenanceWindowComponent())
|
|
735
|
-
```
|
|
736
|
-
|
|
737
|
-
**Built-in components**:
|
|
738
|
-
- `CircuitBreakerComponent` - Prevent runaway agent execution
|
|
739
|
-
- `DeduplicationComponent` - Skip duplicate artifact/agent processing
|
|
740
|
-
|
|
741
|
-
**8 Lifecycle Hooks**: Artifact publication, scheduling decisions, artifact collection, agent scheduling, idle/shutdown
|
|
742
|
-
|
|
743
|
-
---
|
|
744
|
-
|
|
745
|
-
### Production Safety Features
|
|
746
|
-
|
|
747
|
-
**Built-in safeguards prevent common production failures:**
|
|
748
|
-
|
|
749
|
-
```python
|
|
750
|
-
# Circuit breakers prevent runaway costs (via OrchestratorComponent)
|
|
751
|
-
flock = Flock("openai/gpt-4.1") # Auto-adds CircuitBreakerComponent(max_iterations=1000)
|
|
752
|
-
|
|
753
|
-
# Feedback loop protection
|
|
754
|
-
critic = (
|
|
755
|
-
flock.agent("critic")
|
|
756
|
-
.consumes(Essay)
|
|
757
|
-
.publishes(Critique)
|
|
758
|
-
.prevent_self_trigger(True) # Won't trigger itself infinitely
|
|
759
|
-
)
|
|
760
|
-
|
|
761
|
-
# Best-of-N execution (run 5x, pick best)
|
|
762
|
-
agent.best_of(5, score=lambda result: result.metrics["confidence"])
|
|
763
|
-
|
|
764
|
-
# Configuration validation
|
|
765
|
-
agent.best_of(150, ...) # ⚠️ Warns: "best_of(150) is very high - high LLM costs"
|
|
766
|
-
```
|
|
767
|
-
|
|
768
|
-
---
|
|
769
|
-
|
|
770
|
-
## Production-Ready Observability
|
|
771
|
-
|
|
772
|
-
### Sophisticated REST API
|
|
773
|
-
|
|
774
|
-
**Production-ready HTTP endpoints with comprehensive OpenAPI documentation:**
|
|
775
|
-
|
|
776
|
-
Flock includes a fully-featured REST API for programmatic access to the blackboard, agents, and workflow orchestration. Perfect for integration with external systems, building custom UIs, or monitoring production deployments.
|
|
777
|
-
|
|
778
|
-
**Key endpoints:**
|
|
779
|
-
- `POST /api/v1/artifacts` - Publish artifacts to the blackboard
|
|
780
|
-
- `GET /api/v1/artifacts` - Query artifacts with filtering, pagination, and consumption metadata
|
|
781
|
-
- `POST /api/v1/agents/{name}/run` - Direct agent invocation
|
|
782
|
-
- `GET /api/v1/correlations/{correlation_id}/status` - Workflow completion tracking
|
|
783
|
-
- `GET /api/v1/agents` - List all registered agents with subscriptions
|
|
784
|
-
- `GET /health` and `GET /metrics` - Production monitoring
|
|
785
|
-
|
|
786
|
-
**Start the API server:**
|
|
787
|
-
```python
|
|
788
|
-
await flock.serve(dashboard=True) # API + Dashboard on port 8344
|
|
789
|
-
# API docs: http://localhost:8344/docs
|
|
790
|
-
```
|
|
791
|
-
|
|
792
|
-
**Features:**
|
|
793
|
-
- ✅ **OpenAPI 3.0** - Interactive documentation at `/docs`
|
|
794
|
-
- ✅ **Pydantic validation** - Type-safe request/response models
|
|
795
|
-
- ✅ **Correlation tracking** - Monitor workflow completion with polling
|
|
796
|
-
- ✅ **Consumption metadata** - Full artifact lineage and agent execution trails
|
|
797
|
-
- ✅ **Production monitoring** - Health checks and Prometheus-compatible metrics
|
|
798
|
-
|
|
799
|
-
**📖 [Explore the API →](http://localhost:8344/docs)** (start the server first!)
|
|
800
|
-
|
|
801
|
-
### Real-Time Dashboard
|
|
802
|
-
|
|
803
|
-
**Start the dashboard with one line:**
|
|
804
|
-
|
|
805
|
-
```python
|
|
806
|
-
await flock.serve(dashboard=True)
|
|
807
|
-
```
|
|
808
|
-
|
|
809
|
-
The dashboard provides comprehensive real-time visibility into your agent system with professional UI/UX:
|
|
810
|
-
|
|
811
|
-
<p align="center">
|
|
812
|
-
<img alt="Flock Agent View" src="docs/assets/images/flock_ui_agent_view.png" width="1000">
|
|
813
|
-
<i>Agent View: See agent communication patterns and message flows in real-time</i>
|
|
814
|
-
</p>
|
|
815
|
-
|
|
816
|
-
**Key Features:**
|
|
817
|
-
|
|
818
|
-
- **Dual Visualization Modes:**
|
|
819
|
-
- **Agent View** - Agents as nodes with message flows as edges
|
|
820
|
-
- **Blackboard View** - Messages as nodes with data transformations as edges
|
|
821
|
-
|
|
822
|
-
<p align="center">
|
|
823
|
-
<img alt="Flock Blackboard View" src="docs/assets/images/flock_ui_blackboard_view.png" width="1000">
|
|
824
|
-
<i>Blackboard View: Track data lineage and transformations across the system</i>
|
|
825
|
-
</p>
|
|
826
|
-
|
|
827
|
-
- **Real-Time Updates:**
|
|
828
|
-
- WebSocket streaming with 2-minute heartbeat
|
|
829
|
-
- Live agent activation and message publication
|
|
830
|
-
- Auto-layout with Dagre algorithm
|
|
831
|
-
|
|
832
|
-
- **Interactive Graph:**
|
|
833
|
-
- Drag nodes, zoom, pan, and explore topology
|
|
834
|
-
- Double-click nodes to open detail windows
|
|
835
|
-
- Right-click for context menu with auto-layout options:
|
|
836
|
-
- **5 Layout Algorithms**: Hierarchical (Vertical/Horizontal), Circular, Grid, and Random
|
|
837
|
-
- **Smart Spacing**: Dynamic 200px minimum clearance based on node dimensions
|
|
838
|
-
- **Viewport Centering**: Layouts always center around current viewport
|
|
839
|
-
- Add modules dynamically from context menu
|
|
840
|
-
|
|
841
|
-
- **Advanced Filtering:**
|
|
842
|
-
- Correlation ID tracking for workflow tracing
|
|
843
|
-
- Time range filtering (last 5/10/60 minutes or custom)
|
|
844
|
-
- Active filter pills with one-click removal
|
|
845
|
-
- Autocomplete search with metadata preview
|
|
846
|
-
|
|
847
|
-
- **Control Panel:**
|
|
848
|
-
- Publish artifacts from the UI
|
|
849
|
-
- Invoke agents manually
|
|
850
|
-
- Monitor system health
|
|
851
|
-
|
|
852
|
-
- **Keyboard Shortcuts:**
|
|
853
|
-
- `Ctrl+M` - Toggle view mode
|
|
854
|
-
- `Ctrl+F` - Focus filter
|
|
855
|
-
- `Ctrl+/` - Show shortcuts help
|
|
856
|
-
- WCAG 2.1 AA compliant accessibility
|
|
857
|
-
|
|
858
|
-
### Production-Grade Trace Viewer
|
|
859
|
-
|
|
860
|
-
The dashboard includes a **Jaeger-style trace viewer** with 7 powerful visualization modes:
|
|
861
|
-
|
|
862
|
-
<p align="center">
|
|
863
|
-
<img alt="Trace Viewer" src="docs/assets/images/trace_1.png" width="1000">
|
|
864
|
-
<i>Trace Viewer: Timeline view showing span hierarchies and execution flow</i>
|
|
865
|
-
</p>
|
|
866
|
-
|
|
867
|
-
**7 Trace Viewer Modes:**
|
|
868
|
-
|
|
869
|
-
1. **Timeline** - Waterfall visualization with parent-child relationships
|
|
870
|
-
2. **Statistics** - Sortable table view with durations and error tracking
|
|
871
|
-
3. **RED Metrics** - Rate, Errors, Duration monitoring for service health
|
|
872
|
-
4. **Dependencies** - Service-to-service communication analysis
|
|
873
|
-
5. **DuckDB SQL** - Interactive SQL query editor with CSV export
|
|
874
|
-
6. **Configuration** - Real-time service/operation filtering
|
|
875
|
-
7. **Guide** - Built-in documentation and query examples
|
|
876
|
-
|
|
877
|
-
**Additional Features:**
|
|
878
|
-
|
|
879
|
-
- **Full I/O Capture** - Complete input/output data for every operation
|
|
880
|
-
- **JSON Viewer** - Collapsible tree structure with expand all/collapse all
|
|
881
|
-
- **Multi-Trace Support** - Open and compare multiple traces simultaneously
|
|
882
|
-
- **Smart Sorting** - Sort by date, span count, or duration
|
|
883
|
-
- **CSV Export** - Download query results for offline analysis
|
|
884
|
-
|
|
885
|
-
<p align="center">
|
|
886
|
-
<img alt="Trace Viewer" src="docs/assets/images/trace_2.png" width="1000">
|
|
887
|
-
<i>Trace Viewer: Dependency Analysis</i>
|
|
888
|
-
</p>
|
|
889
|
-
|
|
890
|
-
|
|
891
|
-
### OpenTelemetry + DuckDB Tracing
|
|
892
|
-
|
|
893
|
-
**One environment variable enables comprehensive tracing:**
|
|
894
|
-
|
|
895
|
-
```bash
|
|
896
|
-
export FLOCK_AUTO_TRACE=true
|
|
897
|
-
export FLOCK_TRACE_FILE=true
|
|
898
|
-
|
|
899
|
-
python your_app.py
|
|
900
|
-
# Traces stored in .flock/traces.duckdb
|
|
901
|
-
```
|
|
902
|
-
|
|
903
|
-
**AI-queryable debugging:**
|
|
904
|
-
|
|
905
|
-
```python
|
|
906
|
-
import duckdb
|
|
907
|
-
conn = duckdb.connect('.flock/traces.duckdb', read_only=True)
|
|
908
|
-
|
|
909
|
-
# Find bottlenecks
|
|
910
|
-
slow_ops = conn.execute("""
|
|
911
|
-
SELECT name, AVG(duration_ms) as avg_ms, COUNT(*) as count
|
|
912
|
-
FROM spans
|
|
913
|
-
WHERE duration_ms > 1000
|
|
914
|
-
GROUP BY name
|
|
915
|
-
ORDER BY avg_ms DESC
|
|
916
|
-
""").fetchall()
|
|
917
|
-
|
|
918
|
-
# Find errors with full context
|
|
919
|
-
errors = conn.execute("""
|
|
920
|
-
SELECT name, status_description,
|
|
921
|
-
json_extract(attributes, '$.input') as input,
|
|
922
|
-
json_extract(attributes, '$.output') as output
|
|
923
|
-
FROM spans
|
|
924
|
-
WHERE status_code = 'ERROR'
|
|
925
|
-
""").fetchall()
|
|
926
|
-
```
|
|
927
|
-
|
|
928
|
-
**Real debugging session:**
|
|
929
|
-
```
|
|
930
|
-
You: "My pizza agent is slow"
|
|
931
|
-
AI: [queries DuckDB]
|
|
932
|
-
"DSPyEngine.evaluate takes 23s on average.
|
|
933
|
-
Input size: 50KB of conversation history.
|
|
934
|
-
Recommendation: Limit context to last 5 messages."
|
|
935
|
-
```
|
|
936
|
-
|
|
937
|
-
**Why DuckDB?** 10-100x faster than SQLite for analytical queries. Zero configuration. AI agents can debug your AI agents.
|
|
938
|
-
|
|
939
|
-
<p align="center">
|
|
940
|
-
<img alt="Trace Viewer" src="docs/assets/images/trace_3.png" width="1000">
|
|
941
|
-
<i>Trace Viewer: DuckDB Query</i>
|
|
942
|
-
</p>
|
|
943
|
-
|
|
944
|
-
---
|
|
945
|
-
|
|
946
|
-
## Framework Comparison
|
|
947
|
-
|
|
948
|
-
### Architectural Differences
|
|
949
|
-
|
|
950
|
-
Flock uses a fundamentally different coordination pattern than most multi-agent frameworks:
|
|
951
|
-
|
|
952
|
-
| Dimension | Graph-Based Frameworks | Chat-Based Frameworks | Flock (Blackboard) |
|
|
953
|
-
|-----------|------------------------|----------------------|-------------------|
|
|
954
|
-
| **Core Pattern** | Directed graph with explicit edges | Round-robin conversation | Blackboard subscriptions |
|
|
955
|
-
| **Coordination** | Manual edge wiring | Message passing | Type-based subscriptions |
|
|
956
|
-
| **Parallelism** | Manual (split/join nodes) | Sequential turn-taking | Automatic (concurrent consumers) |
|
|
957
|
-
| **Type Safety** | Varies (often TypedDict) | Text-based messages | Pydantic + runtime validation |
|
|
958
|
-
| **Coupling** | Tight (hardcoded successors) | Medium (conversation context) | Loose (type subscriptions only) |
|
|
959
|
-
| **Adding Agents** | Rewrite graph topology | Update conversation flow | Just subscribe to types |
|
|
960
|
-
| **Testing** | Requires full graph | Requires full group | Individual agent isolation |
|
|
961
|
-
| **Security Model** | DIY implementation | DIY implementation | Built-in (5 visibility types) |
|
|
962
|
-
| **Scalability** | O(n²) edge complexity | Limited by turn-taking | O(n) subscription complexity |
|
|
963
|
-
|
|
964
|
-
### When Flock Wins
|
|
965
|
-
|
|
966
|
-
**✅ Use Flock when you need:**
|
|
967
|
-
|
|
968
|
-
- **Parallel agent execution** - Agents consuming the same type run concurrently automatically
|
|
969
|
-
- **Type-safe outputs** - Pydantic validation catches errors at runtime
|
|
970
|
-
- **Minimal prompt engineering** - Schemas define behavior, not natural language
|
|
971
|
-
- **Dynamic agent addition** - Subscribe new agents without rewiring existing workflows
|
|
972
|
-
- **Testing in isolation** - Unit test individual agents with mock inputs
|
|
973
|
-
- **Built-in security** - 5 visibility types for compliance (HIPAA, SOC2, multi-tenancy)
|
|
974
|
-
- **10+ agents** - Linear complexity stays manageable at scale
|
|
975
|
-
|
|
976
|
-
### When Alternatives Win
|
|
977
|
-
|
|
978
|
-
**⚠️ Consider graph-based frameworks when:**
|
|
979
|
-
- You need extensive ecosystem integration with existing tools
|
|
980
|
-
- Your workflow is inherently sequential (no parallelism needed)
|
|
981
|
-
- You want battle-tested maturity (larger communities, more documentation)
|
|
982
|
-
- Your team has existing expertise with those frameworks
|
|
983
|
-
|
|
984
|
-
**⚠️ Consider chat-based frameworks when:**
|
|
985
|
-
- You prefer conversation-based development patterns
|
|
986
|
-
- Your use case maps naturally to turn-taking dialogue
|
|
987
|
-
- You need features specific to those ecosystems
|
|
988
|
-
|
|
989
|
-
### Honest Trade-offs
|
|
990
|
-
|
|
991
|
-
**You trade:**
|
|
992
|
-
- Ecosystem maturity (established frameworks have larger communities)
|
|
993
|
-
- Extensive documentation (we're catching up)
|
|
994
|
-
- Battle-tested age (newer architecture means less production history)
|
|
995
|
-
|
|
996
|
-
**You gain:**
|
|
997
|
-
- Better scalability (O(n) vs O(n²) complexity)
|
|
998
|
-
- Type safety (runtime validation vs hope)
|
|
999
|
-
- Cleaner architecture (loose coupling vs tight graphs)
|
|
1000
|
-
- Production safety (circuit breakers, feedback prevention built-in)
|
|
1001
|
-
- Security model (5 visibility types vs DIY)
|
|
1002
|
-
|
|
1003
|
-
**Different frameworks for different priorities. Choose based on what matters to your team.**
|
|
1004
|
-
|
|
1005
|
-
---
|
|
1006
|
-
|
|
1007
|
-
## Production Readiness
|
|
1008
|
-
|
|
1009
|
-
### What Works Today (v0.5.0)
|
|
1010
|
-
|
|
1011
|
-
**✅ Production-ready core:**
|
|
1012
|
-
- More than 700 tests, with >75% coverage (>90% on critical paths)
|
|
1013
|
-
- Blackboard orchestrator with typed artifacts
|
|
1014
|
-
- Parallel + sequential execution (automatic)
|
|
1015
|
-
- Zero-trust security (5 visibility types)
|
|
1016
|
-
- Circuit breakers and feedback loop prevention
|
|
1017
|
-
- OpenTelemetry distributed tracing with DuckDB storage
|
|
1018
|
-
- Real-time dashboard with 7-mode trace viewer
|
|
1019
|
-
- MCP integration (Model Context Protocol)
|
|
1020
|
-
- Best-of-N execution, batch processing, join operations
|
|
1021
|
-
- Type-safe retrieval API (`get_by_type()`)
|
|
1022
|
-
|
|
1023
|
-
**⚠️ What's missing for large-scale production:**
|
|
1024
|
-
- **Advanced retry logic** - Basic only (exponential backoff planned)
|
|
1025
|
-
- **Event replay** - No Kafka integration yet
|
|
1026
|
-
- **Kubernetes-native deployment** - No Helm chart yet
|
|
1027
|
-
- **OAuth/RBAC** - Dashboard has no auth
|
|
1028
|
-
|
|
1029
|
-
**✅ Available today:**
|
|
1030
|
-
- **Persistent blackboard** - SQLiteBlackboardStore (see above)
|
|
1031
|
-
|
|
1032
|
-
All missing features planned for v1.0
|
|
1033
|
-
|
|
1034
|
-
### Recommended Use Cases Today
|
|
1035
|
-
|
|
1036
|
-
**✅ Good fit right now:**
|
|
1037
|
-
- **Startups/MVPs** - Fast iteration, type safety, built-in observability
|
|
1038
|
-
- **Internal tools** - Where in-memory blackboard is acceptable
|
|
1039
|
-
- **Research/prototyping** - Rapid experimentation with clean architecture
|
|
1040
|
-
- **Medium-scale systems** (10-50 agents, 1000s of artifacts)
|
|
1041
|
-
|
|
1042
|
-
**⚠️ Wait for 1.0 if you need:**
|
|
1043
|
-
- **Enterprise persistence** (multi-region, high availability)
|
|
1044
|
-
- **Compliance auditing** (immutable event logs)
|
|
1045
|
-
- **Multi-tenancy SaaS** (with OAuth/SSO)
|
|
1046
|
-
- **Mission-critical systems** with 99.99% uptime requirements
|
|
1047
|
-
|
|
1048
|
-
**Flock 0.5.0 is production-ready for the right use cases. Know your requirements.**
|
|
1049
|
-
|
|
1050
|
-
---
|
|
1051
|
-
|
|
1052
|
-
## Roadmap to 1.0
|
|
1053
|
-
|
|
1054
|
-
We're building enterprise infrastructure for AI agents and tracking the work publicly. Check [ROADMAP.md](ROADMAP.md) for deep dives and status updates.
|
|
1055
|
-
|
|
1056
|
-
### 0.5.0 Beta (In Flight)
|
|
1057
|
-
- **Core data & governance:** [#271](https://github.com/whiteducksoftware/flock/issues/271), [#274](https://github.com/whiteducksoftware/flock/issues/274), [#273](https://github.com/whiteducksoftware/flock/issues/273), [#281](https://github.com/whiteducksoftware/flock/issues/281)
|
|
1058
|
-
- **Execution patterns & scheduling:** [#282](https://github.com/whiteducksoftware/flock/issues/282), [#283](https://github.com/whiteducksoftware/flock/issues/283)
|
|
1059
|
-
- **REST access & integrations:** [#286](https://github.com/whiteducksoftware/flock/issues/286), [#287](https://github.com/whiteducksoftware/flock/issues/287), [#288](https://github.com/whiteducksoftware/flock/issues/288), [#289](https://github.com/whiteducksoftware/flock/issues/289), [#290](https://github.com/whiteducksoftware/flock/issues/290), [#291](https://github.com/whiteducksoftware/flock/issues/291), [#292](https://github.com/whiteducksoftware/flock/issues/292), [#293](https://github.com/whiteducksoftware/flock/issues/293)
|
|
1060
|
-
- **Docs & onboarding:** [#270](https://github.com/whiteducksoftware/flock/issues/270), [#269](https://github.com/whiteducksoftware/flock/issues/269)
|
|
1061
|
-
|
|
1062
|
-
### 1.0 Release Goals (Target Q4 2025)
|
|
1063
|
-
- **Reliability & operations:** [#277](https://github.com/whiteducksoftware/flock/issues/277), [#278](https://github.com/whiteducksoftware/flock/issues/278), [#279](https://github.com/whiteducksoftware/flock/issues/279), [#294](https://github.com/whiteducksoftware/flock/issues/294)
|
|
1064
|
-
- **Platform validation & quality:** [#275](https://github.com/whiteducksoftware/flock/issues/275), [#276](https://github.com/whiteducksoftware/flock/issues/276), [#284](https://github.com/whiteducksoftware/flock/issues/284), [#285](https://github.com/whiteducksoftware/flock/issues/285)
|
|
1065
|
-
- **Security & access:** [#280](https://github.com/whiteducksoftware/flock/issues/280)
|
|
1066
|
-
|
|
1067
|
-
---
|
|
1068
|
-
|
|
1069
|
-
## Example: Multi-Modal Clinical Decision Support
|
|
1070
|
-
|
|
1071
|
-
```python
|
|
1072
|
-
import os
|
|
1073
|
-
from flock import Flock, flock_type
|
|
1074
|
-
from flock.core.visibility import PrivateVisibility, TenantVisibility, LabelledVisibility
|
|
1075
|
-
from flock.identity import AgentIdentity
|
|
1076
|
-
from pydantic import BaseModel
|
|
1077
|
-
|
|
1078
|
-
@flock_type
|
|
1079
|
-
class PatientScan(BaseModel):
|
|
1080
|
-
patient_id: str
|
|
1081
|
-
scan_type: str
|
|
1082
|
-
image_data: bytes
|
|
1083
|
-
|
|
1084
|
-
@flock_type
|
|
1085
|
-
class XRayAnalysis(BaseModel):
|
|
1086
|
-
findings: list[str]
|
|
1087
|
-
confidence: float
|
|
1088
|
-
|
|
1089
|
-
@flock_type
|
|
1090
|
-
class LabResults(BaseModel):
|
|
1091
|
-
markers: dict[str, float]
|
|
1092
|
-
|
|
1093
|
-
@flock_type
|
|
1094
|
-
class Diagnosis(BaseModel):
|
|
1095
|
-
condition: str
|
|
1096
|
-
reasoning: str
|
|
1097
|
-
confidence: float
|
|
1098
|
-
|
|
1099
|
-
# Create HIPAA-compliant blackboard
|
|
1100
|
-
flock = Flock(os.getenv("DEFAULT_MODEL", "openai/gpt-4.1"))
|
|
1101
|
-
|
|
1102
|
-
# Radiologist with privacy controls
|
|
1103
|
-
radiologist = (
|
|
1104
|
-
flock.agent("radiologist")
|
|
1105
|
-
.consumes(PatientScan)
|
|
1106
|
-
.publishes(
|
|
1107
|
-
XRayAnalysis,
|
|
1108
|
-
visibility=PrivateVisibility(agents={"diagnostician"}) # HIPAA!
|
|
1109
|
-
)
|
|
1110
|
-
)
|
|
1111
|
-
|
|
1112
|
-
# Lab tech with multi-tenancy
|
|
1113
|
-
lab_tech = (
|
|
1114
|
-
flock.agent("lab_tech")
|
|
1115
|
-
.consumes(PatientScan)
|
|
1116
|
-
.publishes(
|
|
1117
|
-
LabResults,
|
|
1118
|
-
visibility=TenantVisibility(tenant_id="patient_123") # Isolation!
|
|
1119
|
-
)
|
|
1120
|
-
)
|
|
1121
|
-
|
|
1122
|
-
# Diagnostician with explicit access
|
|
1123
|
-
diagnostician = (
|
|
1124
|
-
flock.agent("diagnostician")
|
|
1125
|
-
.identity(AgentIdentity(name="diagnostician", labels={"role:physician"}))
|
|
1126
|
-
.consumes(XRayAnalysis, LabResults) # Waits for BOTH
|
|
1127
|
-
.publishes(
|
|
1128
|
-
Diagnosis,
|
|
1129
|
-
visibility=LabelledVisibility(required_labels={"role:physician"})
|
|
1130
|
-
)
|
|
1131
|
-
)
|
|
1132
|
-
|
|
1133
|
-
# Run with tracing
|
|
1134
|
-
async with flock.traced_run("patient_123_diagnosis"):
|
|
1135
|
-
await flock.publish(PatientScan(patient_id="123", ...))
|
|
1136
|
-
await flock.run_until_idle()
|
|
1137
|
-
|
|
1138
|
-
# Get diagnosis (type-safe retrieval)
|
|
1139
|
-
diagnoses = await flock.store.get_by_type(Diagnosis)
|
|
1140
|
-
# Returns list[Diagnosis] directly - no .data access, no casting
|
|
1141
|
-
```
|
|
1142
|
-
|
|
1143
|
-
**What this demonstrates:**
|
|
1144
|
-
- Multi-modal data fusion (images + labs + history)
|
|
1145
|
-
- Built-in access controls (HIPAA compliance)
|
|
1146
|
-
- Parallel agent execution (radiology + labs run concurrently)
|
|
1147
|
-
- Automatic dependency resolution (diagnostician waits for both)
|
|
1148
|
-
- Full audit trail (traced_run + DuckDB storage)
|
|
1149
|
-
- Type-safe data retrieval (no Artifact wrappers)
|
|
1150
|
-
|
|
1151
|
-
---
|
|
1152
|
-
|
|
1153
|
-
## Production Use Cases
|
|
1154
|
-
|
|
1155
|
-
Flock's architecture shines in production scenarios requiring parallel execution, security, and observability. Here are common patterns:
|
|
1156
|
-
|
|
1157
|
-
### Financial Services: Multi-Signal Trading
|
|
1158
|
-
|
|
1159
|
-
**The Challenge:** Analyze multiple market signals in parallel, correlate them within time windows, maintain SEC-compliant audit trails.
|
|
1160
|
-
|
|
1161
|
-
**The Solution:** 20+ signal analyzers run concurrently, join operations correlate signals, DuckDB provides complete audit trails.
|
|
1162
|
-
|
|
1163
|
-
```python
|
|
1164
|
-
# Parallel signal analyzers
|
|
1165
|
-
volatility = flock.agent("volatility").consumes(MarketData).publishes(VolatilityAlert)
|
|
1166
|
-
sentiment = flock.agent("sentiment").consumes(NewsArticle).publishes(SentimentAlert)
|
|
1167
|
-
|
|
1168
|
-
# Trade execution waits for CORRELATED signals (within 5min window)
|
|
1169
|
-
trader = flock.agent("trader").consumes(
|
|
1170
|
-
VolatilityAlert, SentimentAlert,
|
|
1171
|
-
join=JoinSpec(within=timedelta(minutes=5))
|
|
1172
|
-
).publishes(TradeOrder)
|
|
1173
|
-
```
|
|
1174
|
-
|
|
1175
|
-
### Healthcare: HIPAA-Compliant Diagnostics
|
|
1176
|
-
|
|
1177
|
-
**The Challenge:** Multi-modal data fusion with strict access controls, complete audit trails, zero-trust security.
|
|
1178
|
-
|
|
1179
|
-
**The Solution:** Built-in visibility controls for HIPAA compliance, automatic parallel execution, full data lineage tracking.
|
|
1180
|
-
|
|
1181
|
-
```python
|
|
1182
|
-
# Privacy controls built-in
|
|
1183
|
-
radiology.publishes(XRayAnalysis, visibility=PrivateVisibility(agents={"diagnostician"}))
|
|
1184
|
-
lab.publishes(LabResults, visibility=TenantVisibility(tenant_id="patient_123"))
|
|
1185
|
-
|
|
1186
|
-
# Diagnostician waits for BOTH inputs with role-based access
|
|
1187
|
-
diagnostician = flock.agent("diagnostician").consumes(XRayAnalysis, LabResults).publishes(Diagnosis)
|
|
1188
|
-
```
|
|
1189
|
-
|
|
1190
|
-
### E-Commerce: 50-Agent Personalization
|
|
1191
|
-
|
|
1192
|
-
**The Challenge:** Analyze dozens of independent signals, support dynamic signal addition, process millions of events daily.
|
|
1193
|
-
|
|
1194
|
-
**The Solution:** O(n) scaling to 50+ analyzers, batch processing for efficiency, zero graph rewiring when adding signals.
|
|
1195
|
-
|
|
1196
|
-
```python
|
|
1197
|
-
# 50+ signal analyzers (all run in parallel!)
|
|
1198
|
-
for signal in ["browsing", "purchase", "cart", "reviews", "email", "social"]:
|
|
1199
|
-
flock.agent(f"{signal}_analyzer").consumes(UserEvent).publishes(Signal)
|
|
1200
|
-
|
|
1201
|
-
# Recommender batches signals for efficient LLM calls
|
|
1202
|
-
recommender = flock.agent("recommender").consumes(Signal, batch=BatchSpec(size=50))
|
|
1203
|
-
```
|
|
1204
|
-
|
|
1205
|
-
### Multi-Tenant SaaS: Content Moderation
|
|
1206
|
-
|
|
1207
|
-
**The Challenge:** Complete data isolation between tenants, multi-agent consensus, full audit trails.
|
|
1208
|
-
|
|
1209
|
-
**The Solution:** Tenant visibility ensures zero cross-tenant leakage, parallel checks provide diverse signals, traces show complete reasoning.
|
|
1210
|
-
|
|
1211
|
-
**See [USECASES.md](USECASES.md) for complete code examples and production metrics.**
|
|
1212
|
-
|
|
1213
|
-
---
|
|
1214
|
-
|
|
1215
|
-
## Getting Started
|
|
1216
|
-
|
|
1217
|
-
```bash
|
|
1218
|
-
# Install
|
|
1219
|
-
pip install flock-core
|
|
1220
|
-
|
|
1221
|
-
# Set API key
|
|
1222
|
-
export OPENAI_API_KEY="sk-..."
|
|
1223
|
-
|
|
1224
|
-
# Try the examples
|
|
1225
|
-
git clone https://github.com/whiteducksoftware/flock-flow.git
|
|
1226
|
-
cd flock-flow
|
|
1227
|
-
|
|
1228
|
-
# CLI examples with detailed output
|
|
1229
|
-
uv run python examples/01-cli/01_declarative_pizza.py
|
|
1230
|
-
|
|
1231
|
-
# Dashboard examples with visualization
|
|
1232
|
-
uv run python examples/02-dashboard/01_declarative_pizza.py
|
|
1233
|
-
```
|
|
1234
|
-
|
|
1235
|
-
**Learn by doing:**
|
|
1236
|
-
- 📚 [Examples README](examples/README.md) - 12-step learning path from basics to advanced
|
|
1237
|
-
- 🖥️ [CLI Examples](examples/01-cli/) - Detailed console output examples (01-12)
|
|
1238
|
-
- 📊 [Dashboard Examples](examples/02-dashboard/) - Interactive visualization examples (01-12)
|
|
1239
|
-
- 📖 [Documentation](https://whiteducksoftware.github.io/flock) - Complete online documentation
|
|
1240
|
-
- 📘 [AGENTS.md](AGENTS.md) - Development guide
|
|
1241
|
-
|
|
1242
|
-
**Architecture & Patterns:**
|
|
1243
|
-
- 📐 [Architecture Overview](docs/architecture.md) - Understand the refactored codebase structure
|
|
1244
|
-
- 🔧 [Error Handling](docs/patterns/error_handling.md) - Production-ready error patterns
|
|
1245
|
-
- ⚡ [Async Patterns](docs/patterns/async_patterns.md) - Async/await best practices
|
|
1246
|
-
|
|
1247
|
-
---
|
|
1248
|
-
|
|
1249
|
-
## Contributing
|
|
1250
|
-
|
|
1251
|
-
We're building Flock in the open. See **[Contributing Guide](https://whiteducksoftware.github.io/flock/about/contributing/)** for development setup, or check [CONTRIBUTING.md](CONTRIBUTING.md) and [AGENTS.md](AGENTS.md) locally.
|
|
1252
|
-
|
|
1253
|
-
**Before contributing, familiarize yourself with:**
|
|
1254
|
-
- [Architecture Overview](docs/architecture.md) - Codebase organization (Phase 1-7 refactoring)
|
|
1255
|
-
- [Error Handling](docs/patterns/error_handling.md) - Required error patterns
|
|
1256
|
-
- [Async Patterns](docs/patterns/async_patterns.md) - Async/await standards
|
|
1257
|
-
|
|
1258
|
-
**We welcome:**
|
|
1259
|
-
- Bug reports and feature requests
|
|
1260
|
-
- Documentation improvements
|
|
1261
|
-
- Example contributions
|
|
1262
|
-
- Architecture discussions
|
|
1263
|
-
|
|
1264
|
-
**Quality standards:**
|
|
1265
|
-
- All tests must pass
|
|
1266
|
-
- Coverage requirements met
|
|
1267
|
-
- Code formatted with Ruff
|
|
1268
|
-
|
|
1269
|
-
---
|
|
1270
|
-
|
|
1271
|
-
## Why "0.5"?
|
|
1272
|
-
|
|
1273
|
-
We're calling this 0.5 to signal:
|
|
1274
|
-
|
|
1275
|
-
1. **Core is production-ready** - real-world client deployments, comprehensive features
|
|
1276
|
-
2. **Ecosystem is evolving** - Documentation growing, community building, features maturing
|
|
1277
|
-
3. **Architecture is proven** - Blackboard pattern is 50+ years old, declarative contracts are sound
|
|
1278
|
-
4. **Enterprise features are coming** - Persistence, auth, Kubernetes deployment in roadmap
|
|
1279
|
-
|
|
1280
|
-
**1.0 will arrive** when we've delivered persistence, advanced error handling, and enterprise deployment patterns (targeting Q4 2025).
|
|
1281
|
-
|
|
1282
|
-
---
|
|
1283
|
-
|
|
1284
|
-
## The Bottom Line
|
|
1285
|
-
|
|
1286
|
-
**Flock is different because it makes different architectural choices:**
|
|
1287
|
-
|
|
1288
|
-
**Instead of:**
|
|
1289
|
-
- ❌ Prompt engineering → ✅ Declarative type contracts
|
|
1290
|
-
- ❌ Workflow graphs → ✅ Blackboard subscriptions
|
|
1291
|
-
- ❌ Manual parallelization → ✅ Automatic concurrent execution
|
|
1292
|
-
- ❌ Bolt-on security → ✅ Zero-trust visibility controls
|
|
1293
|
-
- ❌ Hope-based debugging → ✅ AI-queryable distributed traces
|
|
1294
|
-
|
|
1295
|
-
**These aren't marketing slogans. They're architectural decisions with real tradeoffs.**
|
|
1296
|
-
|
|
1297
|
-
**You trade:**
|
|
1298
|
-
- Ecosystem maturity (established frameworks have larger communities)
|
|
1299
|
-
- Extensive documentation (we're catching up)
|
|
1300
|
-
- Battle-tested age (newer architecture means less production history)
|
|
1301
|
-
|
|
1302
|
-
**You gain:**
|
|
1303
|
-
- Better scalability (O(n) vs O(n²) complexity)
|
|
1304
|
-
- Type safety (runtime validation vs hope)
|
|
1305
|
-
- Cleaner architecture (loose coupling vs tight graphs)
|
|
1306
|
-
- Production safety (circuit breakers, feedback prevention built-in)
|
|
1307
|
-
- Security model (5 visibility types vs DIY)
|
|
1308
|
-
|
|
1309
|
-
**Different frameworks for different priorities. Choose based on what matters to your team.**
|
|
1310
|
-
|
|
1311
|
-
---
|
|
1312
|
-
|
|
1313
|
-
<div align="center">
|
|
1314
|
-
|
|
1315
|
-
**Built with ❤️ by white duck GmbH**
|
|
1316
|
-
|
|
1317
|
-
**"Declarative contracts eliminate prompt hell. Blackboard architecture eliminates graph spaghetti. Proven patterns applied to modern LLMs."**
|
|
1318
|
-
|
|
1319
|
-
[⭐ Star on GitHub](https://github.com/whiteducksoftware/flock-flow) | [📖 Documentation](https://whiteducksoftware.github.io/flock) | [🚀 Try Examples](examples/) | [💼 Enterprise Support](mailto:support@whiteduck.de)
|
|
1320
|
-
|
|
1321
|
-
</div>
|
|
1322
|
-
|
|
1323
|
-
---
|
|
1324
|
-
|
|
1325
|
-
**Last Updated:** October 19, 2025
|
|
1326
|
-
**Version:** Flock 0.5.0 (Blackboard Edition)
|
|
1327
|
-
**Status:** Production-Ready Core, Enterprise Features Roadmapped
|