rcrewai 0.7.0 → 0.7.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '0024080cd05b151b653bcedb4b29afea563e149f9d68dce1662d636a919020a9'
-  data.tar.gz: 53bb6b1376f88d1379cb0f877eeaa35aed0822f08df5f76d482e28d55101efe6
+  metadata.gz: e0a4a20865a08157cf9148d63439eeb653570c39f4f6e850bf6cc9f8e963a437
+  data.tar.gz: 4abf734ac77a3bdb1c43887b077cd99e1aea4ff9e2f8ba08b5fcac72a27ac7b5
 SHA512:
-  metadata.gz: b7923adbf161caa5361568268306513c92e5ffbddc3736cd91cd0fbccedd8cdd4230854f8310cf358964c7ccea09b1e43040e944f4c5472b0f5e58983bf37aaf
-  data.tar.gz: e4c9426c4e948fca50c3708a8a0e57588b4b18c51a5ac8df0bd2dd902aa2440a20d8d1f6a464333bb9c3244d5ff6901070645bb932fd5fb6e77f3d3f82ff9f83
+  metadata.gz: 01bd0f68ca18dcee76ccca334821bd1412695922b1a84fba092b7bf4686179401c581844b36094b9cf765415e8ce2652cfb7300b496d4c57c9b78dbcd6b7ac22
+  data.tar.gz: afdc22e50a5a61bc0e153c0e8a702e9e0aa465cbf268d06d71a82f372c626236a163ceab777f4eee5e8f6134740a8b54903fdb50cdebd38b6ae7b987d1394581

data/CHANGELOG.md

@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.7.1] - 2026-08-13
+
+### Fixed
+- `Crew#execute` built its event sink but never delivered it to agents, so no agent-level events (`IterationStart`/`IterationEnd`, tool calls, token usage) reached a sink passed to `crew.execute(stream:)`. `Task` now carries a `stream_sink` that `Crew` populates at execute time and passes to `Agent#execute_task`; the two `Process` call sites that bypass `Task#execute` read `crew.stream_sink` directly. Subscribers now receive the full event stream on the sync, async, hierarchical, and consensual paths.
+- Tasks retained a reference to the caller's sink after `execute` returned, keeping request-scoped subscribers reachable for the lifetime of the task object. The sink is now cleared in an `ensure`.
+
+### Note
+- `Events.fan_out` invokes sinks inline on the emitting thread with no serialization, so under `async: true` a sink may be called concurrently from multiple worker threads. Subscribers must do their own locking.
+
 ## [0.7.0] - 2026-07-07
 
 Turns the `:consensual` crew process from a stub into a real multi-agent
@@ -217,7 +226,8 @@ output, guardrails, planning, and training/testing. See `ROADMAP.md`.
 - CLI usage documentation
 - Real-world use cases and examples
 
-[Unreleased]: https://github.com/gkosmo/rcrewAI/compare/v0.7.0...HEAD
+[Unreleased]: https://github.com/gkosmo/rcrewAI/compare/v0.7.1...HEAD
+[0.7.1]: https://github.com/gkosmo/rcrewAI/compare/v0.7.0...v0.7.1
 [0.7.0]: https://github.com/gkosmo/rcrewAI/compare/v0.6.1...v0.7.0
 [0.6.1]: https://github.com/gkosmo/rcrewAI/compare/v0.6.0...v0.6.1
 [0.6.0]: https://github.com/gkosmo/rcrewAI/compare/v0.5.0...v0.6.0

data/docs/api/agent.md

@@ -28,6 +28,14 @@ Creates a new agent instance.
 - `human_input` (Boolean, optional) - Enable human-in-the-loop interactions (default: false)
 - `require_approval_for_tools` (Boolean, optional) - Require human approval for tool usage
 - `require_approval_for_final_answer` (Boolean, optional) - Require human approval for final results
+- `llm` (Symbol \| Hash \| client, optional) - Per-agent LLM override: a provider symbol (`:anthropic`), an options hash (`{ provider:, model:, api_key:, temperature: }`), or a pre-built client. Defaults to the global configuration. See [Advanced Agent Options]({{ site.baseurl }}/tutorials/agent-options)
+- `reasoning` (Boolean, optional) - Run a reasoning/planning pass before answering (default: false); surfaced on the result as `:reasoning`
+- `max_reasoning_attempts` (Integer, optional) - Reasoning retries on empty output (default: 3)
+- `respect_context_window` (Boolean, optional) - Trim history to fit the model's context window (default: false)
+- `max_rpm` (Integer, optional) - Throttle the agent's LLM calls to this many requests per minute (default: unlimited)
+- `knowledge` (Knowledge::Base, optional) - A knowledge base to ground the agent (see [Knowledge (RAG)]({{ site.baseurl }}/tutorials/knowledge))
+- `knowledge_sources` (Array, optional) - Knowledge sources the agent wraps in a base
+- `memory` (Memory \| Hash, optional) - A pre-built `Memory`, or options (`{ embedder:, store:, scope:, short_term_limit:, entity_extractor: }`); defaults to zero-config in-memory (see [Cognitive Memory]({{ site.baseurl }}/tutorials/memory))
 
 **Returns:** `RCrewAI::Agent` instance
 

data/docs/api/crew.md

@@ -10,18 +10,26 @@ The Crew class is the main orchestrator in RCrewAI. It manages a collection of a
 
 ## Class Methods
 
-### `.new(name)`
+### `.new(name, **options)`
 
 Creates a new crew instance.
 
 **Parameters:**
 - `name` (String) - The name of the crew
+- `process` (Symbol, optional) - `:sequential` (default), `:hierarchical`, or `:consensual`
+- `consensus_agents` (Integer, optional) - For `:consensual`, how many agents propose and vote per task (default: 3). See [Consensual Process]({{ site.baseurl }}/tutorials/consensual-process)
+- `planning` (Boolean, optional) - Run a planner pass that drafts a per-task plan before execution (default: false)
+- `planning_llm` (Symbol \| Hash \| client, optional) - The planner's LLM (defaults to the global provider)
+- `knowledge` (Knowledge::Base, optional) - A knowledge base shared with all agents
+- `knowledge_sources` (Array, optional) - Sources the crew wraps in a shared base (see [Knowledge (RAG)]({{ site.baseurl }}/tutorials/knowledge))
+- `verbose` (Boolean, optional) - Detailed logging (default: false)
+- `max_iterations` (Integer, optional) - Max iterations per agent (default: 10)
 
 **Returns:** `RCrewAI::Crew` instance
 
 **Example:**
 ```ruby
-crew = RCrewAI::Crew.new("research_team")
+crew = RCrewAI::Crew.new("research_team", process: :consensual, consensus_agents: 3)
 ```
 
 ### `.create(name)`
@@ -342,4 +350,46 @@ crew.add_agent(manager)
 # Add other agents...
 
 crew.execute
-```
+```
+
+### Consensual Process
+
+Agents propose competing answers and vote to pick the best (see the
+[Consensual Process]({{ site.baseurl }}/tutorials/consensual-process) tutorial).
+
+```ruby
+crew = RCrewAI::Crew.new("panel", process: :consensual, consensus_agents: 3)
+crew.add_agent(junior)
+crew.add_agent(senior)
+crew.add_task(task)
+
+crew.execute   # each task: propose → vote → pick
+```
+
+## Lifecycle, Batch, Training
+
+### `#before_kickoff { |inputs| ... }` / `#after_kickoff { |result| ... }`
+
+Register callbacks that run before/after execution. A `before_kickoff` hook
+receives the inputs hash (from `execute(inputs:)`) and may transform it; an
+`after_kickoff` hook receives and may transform the result. The resolved inputs
+are exposed on `#last_inputs`.
+
+### `#kickoff_for_each(inputs:)`
+
+Runs the crew once per input set, returning one result per input in order. Runs
+are isolated to their own inputs.
+
+```ruby
+results = crew.kickoff_for_each(inputs: [{ topic: "ruby" }, { topic: "python" }])
+```
+
+### `#train(n_iterations:, filename:, feedback: nil)`
+
+Runs the crew repeatedly, collects feedback after each run (via a `feedback:`
+callable, defaulting to a human prompt), and persists it to JSON.
+
+### `#test(n_iterations:, scorer: nil)`
+
+Runs the crew repeatedly and reports per-run and average scores (via a `scorer:`
+callable, defaulting to the run's success rate).

data/docs/api/index.md

@@ -19,23 +19,24 @@ Individual AI agents with specific roles and capabilities.
 ### [RCrewAI::Task]({{ site.baseurl }}/api/task)
 Tasks that agents execute to achieve goals.
 
-## Tools
+## Capabilities (0.4 – 0.7)
+
+These subsystems are covered in depth by the tutorials (with runnable examples):
 
-### [RCrewAI::Tools::WebSearch]({{ site.baseurl }}/api/tools/web-search)
-Enable agents to search the web for information.
+- **`RCrewAI::Flow`** — event-driven workflows. See [Flows]({{ site.baseurl }}/tutorials/flows)
+- **`RCrewAI::Knowledge::Base`** and sources/embedders — RAG. See [Knowledge]({{ site.baseurl }}/tutorials/knowledge)
+- **`RCrewAI::Memory`** (+ `SqliteStore`, memory types) — cognitive memory. See [Memory]({{ site.baseurl }}/tutorials/memory)
+- **Consensual process** — multi-agent voting. See [Consensual Process]({{ site.baseurl }}/tutorials/consensual-process)
+- **Advanced agent/task options** — per-agent LLM, reasoning, rate limiting, context window, multimodal, structured output, guardrails, hooks. See [Advanced Options]({{ site.baseurl }}/tutorials/agent-options)
 
-### [RCrewAI::Tools::FileSystem]({{ site.baseurl }}/api/tools/file-system)
-Allow agents to read and write files.
+## Tools
 
-### [RCrewAI::Tools::Database]({{ site.baseurl }}/api/tools/database)
-Connect agents to databases for data operations.
+See the [Tools System]({{ site.baseurl }}/api/tools) reference for the full API.
+Built-in tools include `WebSearch`, `FileReader`, `FileWriter`, `SqlDatabase`,
+`EmailSender`, `PdfProcessor`, and `CodeExecutor` — plus native function calling
+and MCP servers.
 
 ## Configuration
 
 ### [RCrewAI::Configuration]({{ site.baseurl }}/api/configuration)
-Configure LLM providers and other settings.
-
-## CLI
-
-### [RCrewAI::CLI]({{ site.baseurl }}/api/cli)
-Command-line interface for managing crews.
+Configure LLM providers, models, and other settings.

data/docs/api/task.md

@@ -32,6 +32,13 @@ Creates a new task instance.
 - `allow_guidance` (Boolean) - Allow human guidance during execution
 - `human_review_points` (Array) - Points where human review is requested ([:completion, :error])
 - `async` (Boolean) - Whether task can be executed asynchronously (default: false)
+- `output_schema` (Hash) - A JSON-schema subset the output is validated/coerced against; parsed result on `#structured_output` (raw string on `#raw_result`). Non-conforming output re-runs the agent with the error fed back
+- `guardrail` (callable) - `->(output) { [ok, value_or_error] }` to validate/transform output before it flows downstream
+- `guardrail_max_retries` (Integer) - Guardrail retries with the reason fed back (default: 3)
+- `output_file` (String) - Path to write the result to after completion
+- `create_directory` (Boolean) - Create the output file's parent dirs (default: true)
+- `markdown` (Boolean) - Prepend a heading when the output isn't already markdown (default: false)
+- `attachments` (Array) - Multimodal image inputs, e.g. `[{ type: :image, path: 'x.png' }]` or `{ type: :image, url: '...' }` (OpenAI/Azure). See [Advanced Agent Options]({{ site.baseurl }}/tutorials/agent-options)
 
 **Returns:** `RCrewAI::Task` instance
 

data/docs/examples/async-execution.md

@@ -30,9 +30,10 @@ require 'benchmark'
 RCrewAI.configure do |config|
   config.llm_provider = :openai
   config.temperature = 0.4
-  config.max_concurrent_tasks = 8  # Allow up to 8 concurrent tasks
-  config.task_timeout = 300        # 5-minute timeout per task
+  config.timeout = 300             # request timeout (seconds)
 end
+# Concurrency is a per-run option, not global config:
+#   crew.execute(async: true, max_concurrency: 8)
 
 # ===== CONCURRENT PROCESSING TOOLS =====
 
@@ -885,9 +886,11 @@ context: []  # Dependency management continues with available results
 Designed for horizontal and vertical scaling:
 
 ```ruby
-config.max_concurrent_tasks = 8    # Configurable concurrency
-config.task_timeout = 300          # Timeout management  
-config.resource_pool_size = 16     # Shared resource scaling
+# Concurrency is controlled per run via the async executor:
+crew.execute(async: true, max_concurrency: 8)   # up to 8 concurrent tasks
+
+# Request timeout is global config:
+RCrewAI.configure { |c| c.timeout = 300 }
 ```
 
 This concurrent processing system provides a complete framework for optimizing performance through intelligent parallel execution while maintaining reliability and quality standards across all processing streams.

data/docs/examples/tool-composition.md

@@ -25,12 +25,12 @@ require 'time'
 
 # Configure RCrewAI
 RCrewAI.configure do |config|
-  config.llm_client = :openai
+  config.llm_provider = :openai
   config.openai_api_key = ENV['OPENAI_API_KEY']
   config.log_level = :info
-  config.max_concurrent_tasks = 6
-  config.task_timeout = 300
+  config.timeout = 300              # request timeout (seconds)
 end
+# Concurrency is set per run: crew.execute(async: true, max_concurrency: 6)
 
 # Base tool for common functionality
 class CompositeToolBase < RCrewAI::Tools::Base

data/docs/index.md

@@ -19,16 +19,31 @@ Build powerful AI agent crews in Ruby that work together to accomplish complex t
 ## Features
 
 - **🤖 Intelligent Agents**: AI agents with reasoning loops, memory, and tool usage capabilities
-- **🔗 Multi-LLM Support**: OpenAI, Anthropic (Claude), Google (Gemini), Azure OpenAI, and Ollama
-- **🛠️ Rich Tool Ecosystem**: Web search, file operations, SQL, email, code execution, PDF processing, and custom tools
-- **🧠 Agent Memory**: Short-term and long-term memory for learning from past executions
+- **🔗 Multi-LLM Support**: OpenAI, Anthropic (Claude), Google (Gemini), Azure OpenAI, and Ollama — configurable per agent
+- **🛠️ Rich Tool Ecosystem**: Native function calling, MCP servers, web search, file operations, SQL, email, code execution, PDF processing, and custom tools
+- **🌊 Flows**: Event-driven workflows (`start`/`listen`/`router`) with branching and persistent state
+- **📚 Knowledge (RAG)**: Ground agents in your own documents (string/file/PDF/CSV/URL) with built-in retrieval
+- **🧠 Cognitive Memory**: Semantic recall (embeddings + cosine) with optional SQLite persistence and short-term/long-term/entity/tool memory types
+- **📤 Structured Output & Guardrails**: Schema-validated task output and validate/transform guardrails
+- **🗳️ Flexible Orchestration**: Sequential, hierarchical, and consensual (propose → vote → pick) processes, plus async execution
 - **🤝 Human-in-the-Loop**: Interactive approval workflows, human guidance, and collaborative decision making
-- **⚡ Advanced Task System**: Dependencies, retries, async/concurrent execution, and context sharing
-- **🏗️ Hierarchical Teams**: Manager agents that coordinate and delegate tasks to specialist agents
-- **🔒 Production Ready**: Security controls, error handling, logging, monitoring, and sandboxing
-- **🎯 Flexible Orchestration**: Sequential, hierarchical, and concurrent execution modes
+- **⚡ Advanced Task System**: Dependencies, retries, async/concurrent execution, planning, and context sharing
+- **🎛️ Production Controls**: Rate limiting (`max_rpm`), context-window management, reasoning passes, multimodal input, and streaming events with cost tracking
 - **💎 Ruby-First Design**: Built specifically for Ruby developers with idiomatic patterns
 
+## What's new (0.4 – 0.7)
+
+RCrewAI has expanded well beyond the classic crew model. Recent releases added:
+
+- **[Flows]({{ site.baseurl }}/tutorials/flows)** — a second orchestration pillar for event-driven, stateful workflows
+- **[Knowledge / RAG]({{ site.baseurl }}/tutorials/knowledge)** — retrieval-augmented grounding from your own documents
+- **[Cognitive Memory]({{ site.baseurl }}/tutorials/memory)** — semantic, persistent, multi-type agent memory
+- **[Consensual process]({{ site.baseurl }}/tutorials/consensual-process)** — multi-agent voting to pick the best answer
+- **[Advanced agent options]({{ site.baseurl }}/tutorials/agent-options)** — per-agent LLM, reasoning, rate limiting, context-window management, multimodal, structured output, guardrails, and lifecycle hooks
+
+See the [CHANGELOG](https://github.com/gkosmo/rcrewAI/blob/main/CHANGELOG.md) and the
+per-release upgrade guides (e.g. [upgrading to 0.7](https://github.com/gkosmo/rcrewAI/blob/main/docs/upgrading-to-0.7.md)) for details.
+
 ## Quick Start
 
 ### Basic Agent Collaboration

data/docs/tutorials/agent-options.md

@@ -0,0 +1,128 @@
+---
+layout: tutorial
+title: Advanced Agent & Task Options
+description: Per-agent LLM, reasoning, rate limiting, context window, multimodal, structured output, guardrails, and lifecycle hooks
+---
+
+# Advanced Agent & Task Options
+
+A tour of the production controls added across 0.4–0.7. All are opt-in; agents
+and tasks behave as before when you don't set them.
+
+## Per-agent LLM
+
+Give each agent its own provider/model instead of only the global default:
+
+```ruby
+worker  = RCrewAI::Agent.new(name: 'worker', role: '...', goal: '...',
+                             llm: { provider: :openai, model: 'gpt-4o-mini' })
+manager = RCrewAI::Agent.new(name: 'manager', role: '...', goal: '...',
+                             llm: { provider: :anthropic, model: 'claude-3-opus-20240229' })
+```
+
+Accepts a provider symbol, an options hash, or a pre-built client. Overrides
+never mutate the global configuration.
+
+## Reasoning
+
+Have an agent draft a plan before answering. The trace is exposed on the result
+and doesn't pollute `task.result`:
+
+```ruby
+agent = RCrewAI::Agent.new(name: '...', role: '...', goal: '...',
+                           reasoning: true, max_reasoning_attempts: 3)
+result = agent.execute_task(task)
+result[:reasoning]   # the plan
+result[:content]     # the answer
+```
+
+## Rate limiting
+
+Cap an agent's LLM calls to stay under provider limits (thread-safe, holds under
+async execution):
+
+```ruby
+agent = RCrewAI::Agent.new(name: '...', role: '...', goal: '...', max_rpm: 20)
+```
+
+## Context-window management
+
+Trim history to fit the model's context window (oldest non-system messages drop
+first; system + latest always kept):
+
+```ruby
+agent = RCrewAI::Agent.new(name: '...', role: '...', goal: '...',
+                           respect_context_window: true)
+```
+
+## Multimodal input
+
+Pass images to a vision-capable model via task attachments (local files are
+base64-encoded; URLs pass through). Supported on OpenAI/Azure.
+
+```ruby
+task = RCrewAI::Task.new(
+  name: 'describe', description: 'What is in this chart?', agent: agent,
+  attachments: [
+    { type: :image, path: 'chart.png' },
+    { type: :image, url: 'https://example.com/photo.jpg' }
+  ]
+)
+```
+
+## Structured output & guardrails
+
+Validate, transform, and persist a task's result:
+
+```ruby
+task = RCrewAI::Task.new(
+  name: 'extract', description: '...', agent: agent,
+
+  output_schema: { type: 'object', properties: { title: { type: 'string' } },
+                   required: ['title'] },       # -> task.structured_output
+
+  guardrail: ->(out) { [out.length < 5000, 'too long'] },  # [ok, value_or_error]
+
+  output_file: 'out/report.md', markdown: true
+)
+task.execute
+task.structured_output   # validated object
+task.raw_result          # unprocessed string
+```
+
+Schema/guardrail failures re-run the agent with the error fed back.
+
+## Planning
+
+Run a planner pass that drafts a per-task plan before execution:
+
+```ruby
+crew = RCrewAI::Crew.new('research', planning: true)   # optional planning_llm:
+```
+
+## Lifecycle hooks & batch runs
+
+```ruby
+crew.before_kickoff { |inputs| inputs.merge(started_at: Time.now) }
+crew.after_kickoff  { |result| notify(result); result }
+
+crew.execute(inputs: { topic: 'ruby' })
+crew.last_inputs   # the resolved inputs
+
+# Run the crew once per input set:
+results = crew.kickoff_for_each(inputs: [{ topic: 'ruby' }, { topic: 'python' }])
+```
+
+## Training & testing
+
+```ruby
+crew.train(n_iterations: 3, filename: 'training.json')  # collect feedback
+crew.test(n_iterations: 5)                              # score repeated runs
+```
+
+## See also
+
+- [Flows]({{ site.baseurl }}/tutorials/flows)
+- [Knowledge (RAG)]({{ site.baseurl }}/tutorials/knowledge)
+- [Cognitive Memory]({{ site.baseurl }}/tutorials/memory)
+- [Consensual Process]({{ site.baseurl }}/tutorials/consensual-process)

data/docs/tutorials/consensual-process.md

@@ -0,0 +1,58 @@
+---
+layout: tutorial
+title: Consensual Process
+description: Multi-agent consensus — agents propose competing answers and vote to pick one
+---
+
+# Consensual Process
+
+For decisions where multiple perspectives matter, the `:consensual` process has
+several agents propose competing answers and vote to pick the best one.
+
+> **Since 0.7.0.** Earlier versions treated `:consensual` as a stub that ran
+> tasks sequentially. It now performs real consensus — if you relied on the old
+> behavior, use `process: :sequential`.
+
+## How it works
+
+For each task:
+
+1. **Propose** — up to `consensus_agents` agents (default 3, capped from the
+   crew) each produce a candidate answer.
+2. **Vote** — every participant scores each candidate 0–10 against the task's
+   description and expected output.
+3. **Pick** — the highest total score wins. Ties break toward the task's
+   assigned agent.
+
+```ruby
+crew = RCrewAI::Crew.new('panel', process: :consensual, consensus_agents: 3)
+crew.add_agent(junior)
+crew.add_agent(senior)
+crew.add_task(task)
+
+result = crew.execute   # each task goes through propose → vote → pick
+```
+
+## Cost
+
+Consensus multiplies LLM calls: roughly `N` proposals + `N × N` scoring calls per
+task, where `N` is `consensus_agents` (default 3). The cap keeps cost bounded even
+on large crews — raise or lower it to trade thoroughness for cost.
+
+## Edge cases
+
+- **One agent** → a single proposal (no meaningful vote), still a valid result.
+- **A proposer errors** → that candidate is dropped; consensus continues with the
+  rest.
+- **All proposals fail** → the task is marked failed.
+
+## When to use it
+
+Reach for `:consensual` when answer quality benefits from diversity and
+cross-checking — design decisions, judgment calls, ambiguous tasks. For
+straightforward pipelines, `:sequential` or `:hierarchical` is cheaper.
+
+## Runnable example
+
+See [`examples/consensual_process_example.rb`](https://github.com/gkosmo/rcrewAI/blob/main/examples/consensual_process_example.rb)
+— runs without an API key.

data/docs/tutorials/flows.md

@@ -0,0 +1,135 @@
+---
+layout: tutorial
+title: Flows — Event-Driven Workflows
+description: Build structured, stateful workflows with start/listen/router, combinators, and persistence
+---
+
+# Flows
+
+Crews are great for "have these agents produce these outputs." **Flows** are the
+second orchestration pillar — for workflows that need explicit branching, joins,
+persistent state, or coordination across multiple crews and plain Ruby steps.
+
+Subclass `RCrewAI::Flow` and wire methods together with a class-level DSL.
+
+## A first flow
+
+```ruby
+require 'rcrewai'
+
+class ArticleFlow < RCrewAI::Flow
+  start :outline
+  def outline
+    state.sections = %w[intro body conclusion]
+    state.sections.length          # return value is passed to listeners of :outline
+  end
+
+  listen :outline
+  def draft(section_count)
+    state.words = section_count * 100
+    state.words
+  end
+
+  router :draft
+  def review(words)
+    words >= 250 ? :publish : :expand   # a router returns a label
+  end
+
+  listen :publish
+  def publish = state.status = 'published'
+
+  listen :expand
+  def expand = state.status = 'needs more work'
+end
+
+flow = ArticleFlow.new
+flow.kickoff(inputs: { author: 'Ada' })
+flow.state.status   # => "published"
+flow.state.id       # => automatic UUID
+```
+
+## The DSL
+
+- **`start :method`** — an entry point. A flow can have several; all run first.
+- **`listen :trigger`** — runs the following method after `:trigger` completes,
+  receiving its return value.
+- **`router :trigger`** — like `listen`, but the method's return value becomes a
+  **label** that other `listen` methods can trigger on. This is how you branch.
+
+### Combining triggers
+
+```ruby
+listen and_(:fetch_a, :fetch_b)   # fires once, after BOTH complete
+def merge(...); end
+
+listen or_(:cache_hit, :cache_miss)  # fires when EITHER completes
+def proceed(...); end
+```
+
+## State
+
+`state` is a schemaless object with an automatic UUID. Read and write attributes
+directly (`state.foo = 1`), and seed initial values via `kickoff(inputs:)`:
+
+```ruby
+flow.kickoff(inputs: { topic: 'ruby', max_words: 800 })
+flow.state.topic       # => "ruby"
+```
+
+## Persistence — pause and resume
+
+Pass a `state_store:` and a flow's state is saved after each run, so you can
+restore it later by id:
+
+```ruby
+store = RCrewAI::Flow::FileStateStore.new('tmp/flows')  # or your own #save/#load
+flow  = ArticleFlow.new(state_store: store)
+flow.kickoff
+id = flow.state.id
+
+# ...later, even in a fresh process...
+resumed = ArticleFlow.new(state_store: store)
+resumed.restore(id)
+resumed.state.status   # => recovered
+```
+
+Built-in stores: `RCrewAI::Flow::MemoryStateStore` (volatile) and
+`RCrewAI::Flow::FileStateStore` (JSON on disk). Any object responding to
+`#save(id, hash)` / `#load(id)` works.
+
+## Running a crew inside a flow
+
+A flow step is just a method, so it can kick off a whole crew:
+
+```ruby
+class ResearchFlow < RCrewAI::Flow
+  def initialize(crew:, **opts)
+    super(**opts)
+    @crew = crew
+  end
+
+  start :run
+  def run
+    state.crew_result = @crew.execute(inputs: { topic: state.topic })
+  end
+end
+```
+
+## Human feedback
+
+Pause a flow for input with `human_feedback`:
+
+```ruby
+listen :draft
+def approve(_draft)
+  state.approved = human_feedback('Approve this draft?')
+end
+```
+
+Provide a handler for non-interactive runs:
+`ArticleFlow.new(feedback_handler: ->(prompt) { auto_approve(prompt) })`.
+
+## Runnable example
+
+See [`examples/flow_example.rb`](https://github.com/gkosmo/rcrewAI/blob/main/examples/flow_example.rb)
+— it runs without an API key.

data/docs/tutorials/index.md

@@ -62,6 +62,51 @@ Welcome to the RCrewAI tutorials! These step-by-step guides will take you from b
 
 ---
 
+## ✨ Capabilities (0.4 – 0.7)
+
+The features that grew RCrewAI beyond the classic crew model.
+
+### [Flows — Event-Driven Workflows]({{ site.baseurl }}/tutorials/flows)
+**Orchestrate with branching and state.** `start`/`listen`/`router`, `and_`/`or_`
+combinators, schemaless state with a UUID, persistence and resume, running crews
+as steps, and `human_feedback` pause points.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Intermediate ⭐⭐
+
+### [Knowledge (RAG)]({{ site.baseurl }}/tutorials/knowledge)
+**Ground agents in your documents.** String/file/PDF/CSV/URL sources, chunking,
+multi-provider embeddings, and agent- or crew-level attachment with automatic
+retrieval into the prompt.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Intermediate ⭐⭐
+
+### [Cognitive Memory]({{ site.baseurl }}/tutorials/memory)
+**Agents that remember.** Semantic recall (embeddings + cosine), optional SQLite
+persistence, and short-term/long-term/entity/tool memory types — with a
+zero-config default.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Intermediate ⭐⭐
+
+### [Consensual Process]({{ site.baseurl }}/tutorials/consensual-process)
+**Multi-agent voting.** Agents propose competing answers and score each other to
+pick the best; tune cost with `consensus_agents`.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Beginner ⭐
+
+### [Advanced Agent & Task Options]({{ site.baseurl }}/tutorials/agent-options)
+**Production controls.** Per-agent LLM, reasoning passes, rate limiting,
+context-window management, multimodal input, structured output, guardrails,
+planning, lifecycle hooks, and batch runs.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Intermediate ⭐⭐
+
+---
+
 ## 🏗️ Architecture & Scaling
 
 ### [Working with Multiple Crews]({{ site.baseurl }}/tutorials/multiple-crews)

data/docs/tutorials/knowledge.md

@@ -0,0 +1,80 @@
+---
+layout: tutorial
+title: Knowledge (RAG)
+description: Ground agents in your own documents with retrieval-augmented generation
+---
+
+# Knowledge (RAG)
+
+Give agents access to your own documents. Sources are chunked, embedded, and
+stored in a vector store; at execution time the most relevant chunks are
+injected into the agent's task prompt automatically.
+
+## Building a knowledge base
+
+```ruby
+require 'rcrewai'
+
+kb = RCrewAI::Knowledge::Base.new(sources: [
+  RCrewAI::Knowledge::StringSource.new('Refunds are available within 30 days.'),
+  RCrewAI::Knowledge::FileSource.new('docs/policy.txt'),
+  RCrewAI::Knowledge::PdfSource.new('handbook.pdf'),
+  RCrewAI::Knowledge::CsvSource.new('faq.csv'),
+  RCrewAI::Knowledge::UrlSource.new('https://example.com/faq')
+])
+```
+
+## Attaching knowledge
+
+**Agent-level** (role-specific):
+
+```ruby
+support = RCrewAI::Agent.new(
+  name: 'support', role: 'Support specialist', goal: 'Answer using company policy',
+  knowledge: kb
+)
+
+# Or pass raw sources and let the agent build the base:
+support = RCrewAI::Agent.new(name: 'support', role: '...', goal: '...',
+                             knowledge_sources: [RCrewAI::Knowledge::StringSource.new('...')])
+```
+
+**Crew-level** (shared with every agent):
+
+```ruby
+crew = RCrewAI::Crew.new('support_crew', knowledge: kb)
+```
+
+When a task runs, chunks relevant to the task description are retrieved and added
+to the prompt under a "Relevant Knowledge" heading.
+
+## Embeddings — pick a provider
+
+Embeddings default to OpenAI's `text-embedding-3-small`. Since 0.6.1 the embedder
+is multi-provider:
+
+```ruby
+# Local, no API key:
+embedder = RCrewAI::Knowledge::Embedder.new(provider: :ollama, model: 'nomic-embed-text')
+
+# Or :azure / :google. (:anthropic has no embeddings API and raises.)
+kb = RCrewAI::Knowledge::Base.new(sources: [...], embedder: embedder)
+```
+
+Any object responding to `embed(texts) -> [[float, ...], ...]` can be substituted.
+
+## Chunking and the vector store
+
+`Knowledge::Base.new` accepts `chunk_size:` and `overlap:` to tune how documents
+are split. The default vector store is in-memory with cosine similarity; the
+store is pluggable if you need a different backend.
+
+```ruby
+kb = RCrewAI::Knowledge::Base.new(sources: [...], chunk_size: 800, overlap: 100)
+kb.search('what is the refund window?', k: 3)   # => top-k relevant chunks
+```
+
+## Runnable example
+
+See [`examples/knowledge_rag_example.rb`](https://github.com/gkosmo/rcrewAI/blob/main/examples/knowledge_rag_example.rb)
+— it runs without an API key using a fake embedder.

data/docs/tutorials/memory.md

@@ -0,0 +1,90 @@
+---
+layout: tutorial
+title: Cognitive Memory
+description: Semantic, persistent, multi-type agent memory
+---
+
+# Cognitive Memory
+
+Agents remember what they've done and recall it on future tasks. Memory is
+**zero-config by default** (in-memory, word-overlap recall) and becomes far more
+capable when you add an embedder (semantic recall) and a store (persistence).
+
+## Zero-config
+
+Every agent gets a `Memory` scoped to itself. Nothing to set up:
+
+```ruby
+agent = RCrewAI::Agent.new(name: 'engineer', role: '...', goal: '...')
+# agent.memory records executions and recalls relevant ones automatically
+```
+
+## Semantic recall
+
+Pass an embedder and recall becomes semantic — the agent finds conceptually
+related past work even when the wording differs:
+
+```ruby
+embedder = RCrewAI::Knowledge::Embedder.new           # or provider: :ollama
+agent = RCrewAI::Agent.new(name: 'engineer', role: '...', goal: '...',
+                           memory: { embedder: embedder })
+```
+
+Recall falls back to word-overlap similarity without an embedder, and embedding
+failures fall back gracefully — **memory never breaks agent execution.**
+
+## Persistence
+
+Give memory a SQLite store and it survives restarts:
+
+```ruby
+store = RCrewAI::Memory::SqliteStore.new(path: '~/.rcrewai/memory.db')
+agent = RCrewAI::Agent.new(name: 'engineer', role: '...', goal: '...',
+                           memory: { embedder: embedder, store: store })
+```
+
+The default store is `InMemoryStore` (volatile). `SqliteStore` accepts
+`max_candidates:` (default 1000) to bound how many recent rows a search scans,
+keeping recall fast as memory grows.
+
+## Memory types
+
+The `Memory` facade exposes four underlying types:
+
+```ruby
+agent.memory.short_term   # recent executions (capped, semantic recall)
+agent.memory.long_term    # durable, deduped insights from successful runs
+agent.memory.entity       # facts about entities (people, systems) seen in work
+agent.memory.tool         # tool-call history + outcomes
+
+agent.memory.entity.entities              # => ["Alice", "AWS", ...]
+agent.memory.long_term.recall('...', limit: 3)
+```
+
+### Better entity extraction
+
+By default entities are extracted heuristically (capitalized tokens). For
+multi-word names, plug in an LLM extractor:
+
+```ruby
+extractor = RCrewAI::Memory::LlmEntityExtractor.new(agent.llm_client)
+agent = RCrewAI::Agent.new(name: '...', role: '...', goal: '...',
+                           memory: { entity_extractor: extractor })
+```
+
+## Scoping
+
+Memory is scoped per agent, so agents sharing a persistent store don't read each
+other's memories. Override with `memory: { scope: 'shared' }` for deliberate
+sharing.
+
+## The classic API still works
+
+`add_execution`, `add_tool_usage`, `relevant_executions`, `tool_usage_for`,
+`clear_short_term!`, `clear_all!`, and `stats` behave as before — the cognitive
+system is a drop-in upgrade.
+
+## Runnable example
+
+See [`examples/cognitive_memory_example.rb`](https://github.com/gkosmo/rcrewAI/blob/main/examples/cognitive_memory_example.rb)
+— semantic recall + SQLite persistence, no API key required.

data/lib/rcrewai/crew.rb

@@ -64,6 +64,7 @@ module RCrewAI
       sinks << block if block_given?
       Array(stream).each { |s| sinks << s } if stream
       @stream_sink = sinks.empty? ? nil : RCrewAI::Events.fan_out(sinks)
+      @tasks.each { |t| t.stream_sink = @stream_sink }
 
       run_before_hooks(inputs)
 
@@ -72,6 +73,11 @@ module RCrewAI
 
       result = async ? execute_async(**async_options) : execute_sync
       run_after_hooks(result)
+    ensure
+      # Drop per-task references to the caller's sink so request-scoped
+      # collectors do not stay reachable after the run. The crew's own
+      # +stream_sink+ reader is left intact: Process reads it.
+      @tasks.each { |t| t.stream_sink = nil }
     end
 
     # Runs the crew once per input set, returning one result per input in order.

data/lib/rcrewai/process.rb

@@ -351,7 +351,7 @@ module RCrewAI
         end
 
         # Execute the task
-        agent.execute_task(enhanced_task)
+        agent.execute_task(enhanced_task, stream: crew.stream_sink)
       end
 
       def should_abort_execution?(failed_tasks, phase_number, _plan)
@@ -421,7 +421,7 @@ module RCrewAI
 
       def gather_proposals(task, participants)
         participants.filter_map do |agent|
-          content = extract_content(agent.execute_task(task))
+          content = extract_content(agent.execute_task(task, stream: crew.stream_sink))
           { agent: agent, content: content }
         rescue StandardError => e
           @logger.warn "Agent #{agent.name} failed to propose: #{e.message}"

data/lib/rcrewai/task.rb

@@ -10,7 +10,7 @@ module RCrewAI
     include HumanInteractionExtensions
     attr_reader :name, :description, :agent, :context, :expected_output, :tools, :async,
                 :raw_result, :structured_output, :attachments
-    attr_accessor :result, :status, :start_time, :end_time, :execution_time
+    attr_accessor :result, :status, :start_time, :end_time, :execution_time, :stream_sink
 
     def initialize(name:, description:, agent: nil, **options)
       @name = name
@@ -44,6 +44,7 @@ module RCrewAI
       @start_time = nil
       @end_time = nil
       @execution_time = nil
+      @stream_sink = nil
       @retry_count = 0
       @max_retries = options.fetch(:max_retries, 2)
     end
@@ -218,7 +219,7 @@ module RCrewAI
 
       loop do
         attempts += 1
-        raw = extract_content(agent.execute_task(self))
+        raw = extract_content(agent.execute_task(self, stream: @stream_sink))
         @raw_result = raw
 
         begin

data/lib/rcrewai/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RCrewAI
-  VERSION = '0.7.0'
+  VERSION = '0.7.1'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rcrewai
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.7.1
 platform: ruby
 authors:
 - gkosmo
@@ -359,10 +359,15 @@ files:
 - docs/superpowers/specs/2026-07-06-cognitive-memory-design.md
 - docs/superpowers/specs/2026-07-07-consensual-process-design.md
 - docs/tutorials/advanced-agents.md
+- docs/tutorials/agent-options.md
+- docs/tutorials/consensual-process.md
 - docs/tutorials/custom-tools.md
 - docs/tutorials/deployment.md
+- docs/tutorials/flows.md
 - docs/tutorials/getting-started.md
 - docs/tutorials/index.md
+- docs/tutorials/knowledge.md
+- docs/tutorials/memory.md
 - docs/tutorials/multiple-crews.md
 - docs/upgrading-to-0.3.md
 - docs/upgrading-to-0.4.md