rcrewai 0.3.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c7682076eeeb0d3c1bdc0de2185c83ad8c925ce19165786daa0910c1925fdf64
-  data.tar.gz: 1db4ba5508d8aef52b6645be9cae3dcb7328564f8335a24fe844655196d2f53f
+  metadata.gz: 68a381d21e047e37972fda09fd7e9faa93da77e6fe01a040a4c77b9ee0066aca
+  data.tar.gz: 3818acc6b9b6eb71d27d4099cb1c1fc574adbdd0a3a8128389803c051c364774
 SHA512:
-  metadata.gz: dfaf95b581c86cd36573400729a02e72e307721f51267fefafdb3245aae28bffc89201855dcd294f38ecfc54a6a1dbf2062d92fbd55310e4f2f9d968b49a1a6e
-  data.tar.gz: 4c8ed1132d754e92ab154c192f28d8add3d80fdbedbaaa15fa6fd876002f18674d30a91f7ef9fb5dcba65a4240ca9f464d26fa15b64a8cdcdee4ad806cefe3bd
+  metadata.gz: 65d796bde314db55466f7f4f043db9eeb772dee3e72914d2bc27947eb6dfe84ce246b89a6181d473c3f7d175d353fbc9531914c8baa5b7b010e02229324bac27
+  data.tar.gz: b2fffaab35dc672b12d2a5aea5fd0b02e6519b2f4a5ca98daaf66bef740551e2e674f4cc39967f3d987ed9dfe1c13e7dec4c8a30e5b86e2140242e347dd96a11

data/.rubocop.yml

@@ -1 +1,21 @@
 inherit_from: .rubocop_todo.yml
+
+Naming/MethodParameterName:
+  # `k` (top-k retrieval) and short math vars for vector similarity are
+  # conventional and clearer than forced longer names. The rest are RuboCop's
+  # defaults, restated because AllowedNames replaces rather than extends.
+  AllowedNames:
+    - k
+    - a
+    - b
+    - io
+    - id
+    - to
+    - by
+    - 'on'
+    - in
+    - at
+    - ip
+    - db
+    - os
+    - pp

data/CHANGELOG.md

@@ -7,6 +7,57 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.5.0] - 2026-07-03
+
+Polish release completing the roadmap backlog: crew lifecycle hooks, batch
+execution, rate limiting, per-agent reasoning, context-window management, and
+multimodal image input. All additive — existing code runs unchanged.
+
+### Added
+- Crew lifecycle hooks: `Crew#before_kickoff` and `Crew#after_kickoff` register callbacks that run before/after execution. A `before_kickoff` hook receives the inputs hash (passed via `crew.execute(inputs:)`) and may transform it; an `after_kickoff` hook receives the result and may transform it. Multiple hooks run in registration order. The (possibly transformed) inputs are exposed on `Crew#last_inputs`. (#15)
+- `Crew#kickoff_for_each(inputs:)` runs the crew once per input set and returns one result per input, in order. Runs are isolated — each execution starts from only its own inputs. (#16)
+- Rate limiting: `Agent.new(max_rpm:)` throttles the agent's LLM calls to the given requests-per-minute using a thread-safe rolling-window `RCrewAI::RateLimiter`. The agent's client is transparently wrapped (`RateLimiter::ThrottledClient`) so every `chat` acquires a slot first; `max_rpm` nil/0 means unlimited. (#17)
+- Reasoning: `Agent.new(reasoning: true)` runs a reasoning/planning pass before answering — the LLM drafts a short plan for the task, which is injected into the answer prompt and surfaced on the result hash as `:reasoning` (without polluting `task.result`). Bounded by `max_reasoning_attempts` (default 3), retrying if the model returns empty output; if every attempt is empty, execution proceeds without a plan. Off by default. (#18)
+- Context-window management: `Agent.new(respect_context_window: true)` trims the message history to fit the model's context window before each LLM call, dropping the oldest non-system messages while always keeping system messages and the latest message. The new `RCrewAI::ContextWindow` module provides the token estimate (chars/4 heuristic), a per-model window-size table, and the `fit` trimmer. Off by default. (#19)
+- Multimodal input: `Task.new(attachments:)` accepts image attachments (`{ type: :image, url: '...' }` or `{ type: :image, path: '...' }`). When a task has attachments, the agent builds an OpenAI-style multimodal user message (text + `image_url` parts); local files are base64-encoded into data URLs with a mime type inferred from the extension. Supported on OpenAI/Azure; other providers raise a clear `Multimodal::UnsupportedProviderError`. The new `RCrewAI::Multimodal` module builds the content parts. (#20)
+
+## [0.4.0] - 2026-07-03
+
+This release closes the feature-parity gap with the modern CrewAI framework,
+adding its second pillar (**Flows**) alongside **Knowledge (RAG)**, structured
+output, guardrails, planning, and training/testing. See `ROADMAP.md`.
+
+### Added
+
+#### Flows (#11)
+- `RCrewAI::Flow` — an event-driven workflow engine (CrewAI's second pillar). Subclass it and declare methods with a class-level DSL: `start`, `listen`, `router`, and the `and_` / `or_` trigger combinators. `kickoff` runs the graph to a fixed point; routers emit labels that listeners trigger on.
+- Flow state (`Flow::State`) is a schemaless object with an automatic UUID, seedable via `kickoff(inputs:)`.
+- Flow persistence: pluggable state stores (`Flow::MemoryStateStore`, `Flow::FileStateStore`, or any `#save`/`#load` object); `flow.restore(id)` resumes a persisted run.
+- Flows can invoke a `Crew` as a step and pause for input via `#human_feedback`.
+
+#### Knowledge / RAG (#9)
+- `RCrewAI::Knowledge` module adds retrieval-augmented context. Sources (`StringSource`, `FileSource`, `PdfSource`, `CsvSource`, `UrlSource`) are chunked, embedded, and stored in an in-memory cosine-similarity vector store (no external DB required).
+- Attach via `Agent.new(knowledge:)` / `knowledge_sources:` (role-specific) or `Crew.new(knowledge:)` / `knowledge_sources:` (shared with all agents); relevant chunks are injected into each task's prompt at execution.
+- The embedder (`Knowledge::Embedder`, default OpenAI `text-embedding-3-small`) and vector store are pluggable.
+
+#### Task output processing (#6, #7, #8)
+- Structured output: `Task.new(output_schema:)` validates and coerces the agent's output against a JSON-schema subset, exposing the parsed object via `Task#structured_output` (and the raw string via `Task#raw_result`). JSON embedded in surrounding prose or a fenced code block is extracted automatically; output that doesn't conform re-runs the agent with the error fed back.
+- Guardrails: `Task.new(guardrail:)` takes a callable returning `[ok, value_or_error]` to validate and transform output before it flows downstream, retrying up to `guardrail_max_retries` (default 3) with the rejection reason fed back to the agent.
+- Output persistence & formatting: `Task.new(output_file:)` writes the result to disk (`create_directory:` controls parent-dir creation, default true), and `markdown: true` prepends a heading when the output isn't already a markdown document.
+- `RCrewAI::OutputSchema` — a small JSON-schema-subset validator/coercer used by structured task output.
+
+#### Per-agent LLM (#5)
+- `Agent.new(llm:)` accepts a provider symbol (`:anthropic`), an options hash (`{ provider:, model:, api_key:, temperature: }`), or a pre-built client instance. Agents in the same crew can use different providers/models (e.g. a cheap worker model and a stronger manager model). Omitting `llm:` keeps the previous global-configuration behavior.
+- `Configuration#with_overrides` returns a copy of the configuration with per-agent overrides applied, leaving global state untouched.
+
+#### Planning (#10)
+- `Crew.new(planning: true)` runs a single planner pass before execution that asks an LLM to draft a short plan for each task and folds it into the task's description. Optional `planning_llm:` selects the planner client (defaults to the global provider). Best-effort — a planner error or unparseable output leaves tasks unchanged and execution proceeds.
+- `Task#enrich_description` appends supplementary guidance (used by the planner) without discarding the original instructions.
+
+#### Training & testing (#12)
+- `Crew#train(n_iterations:, filename:)` runs the crew repeatedly, collects feedback after each iteration (via a `feedback:` callable, defaulting to a human prompt), and persists it as JSON.
+- `Crew#test(n_iterations:)` runs the crew repeatedly and reports per-run and average scores (via a `scorer:` callable, defaulting to the run's success rate).
+
 ## [0.3.0] - 2026-05-12
 
 ### Added
@@ -128,5 +179,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - CLI usage documentation
 - Real-world use cases and examples
 
-[Unreleased]: https://github.com/gkosmo/rcrewAI/compare/v0.1.0...HEAD
+[Unreleased]: https://github.com/gkosmo/rcrewAI/compare/v0.5.0...HEAD
+[0.5.0]: https://github.com/gkosmo/rcrewAI/compare/v0.4.0...v0.5.0
+[0.4.0]: https://github.com/gkosmo/rcrewAI/compare/v0.3.0...v0.4.0
+[0.3.0]: https://github.com/gkosmo/rcrewAI/compare/v0.1.0...v0.3.0
 [0.1.0]: https://github.com/gkosmo/rcrewAI/releases/tag/v0.1.0

data/README.md

@@ -19,6 +19,8 @@ RCrewAI is a Ruby implementation of the CrewAI framework, allowing you to create
 - **🏗️ Hierarchical Teams**: Manager agents that coordinate and delegate tasks to specialist agents
 - **🔒 Production Ready**: Security controls, error handling, logging, monitoring, and sandboxing
 - **🎯 Flexible Orchestration**: Sequential, hierarchical, and concurrent execution modes
+- **🌊 Flows**: Event-driven workflows with `start`/`listen`/`router`, branching, and persistent state
+- **📚 Knowledge (RAG)**: Ground agents in your own documents with built-in retrieval
 - **💎 Ruby-First Design**: Built specifically for Ruby developers with idiomatic patterns
 
 ## 📦 Installation
@@ -169,6 +171,254 @@ RCrewAI.configure do |config|
 end
 ```
 
+### Per-agent LLM
+
+The `RCrewAI.configure` block sets the crew-wide default. Any agent can override
+it with the `llm:` option, so a single crew can mix providers and models — for
+example a cheap model for workers and a stronger one for the manager:
+
+```ruby
+# Provider only (uses that provider's configured model + key)
+researcher = RCrewAI::Agent.new(name: 'researcher', role: '...', goal: '...',
+                                llm: :anthropic)
+
+# Provider + model (and optionally api_key / temperature)
+manager = RCrewAI::Agent.new(name: 'manager', role: '...', goal: '...',
+                             llm: { provider: :anthropic, model: 'claude-3-opus-20240229' })
+
+worker = RCrewAI::Agent.new(name: 'worker', role: '...', goal: '...',
+                            llm: { provider: :openai, model: 'gpt-4o-mini' })
+
+# Or pass a pre-built client instance
+worker = RCrewAI::Agent.new(name: 'worker', role: '...', goal: '...',
+                            llm: my_client)
+```
+
+Omit `llm:` to use the global `RCrewAI.configure` settings. Overrides never
+mutate the global configuration.
+
+## 📤 Structured Output, Guardrails & File Output
+
+Tasks can validate, transform, and persist their output:
+
+```ruby
+task = RCrewAI::Task.new(
+  name: 'extract',
+  description: 'Extract the article title and word count as JSON',
+  agent: analyst,
+
+  # Structured output — validated & coerced against a JSON schema.
+  # Non-conforming output re-runs the agent with the error fed back.
+  output_schema: {
+    type: 'object',
+    properties: { title: { type: 'string' }, words: { type: 'integer' } },
+    required: ['title']
+  },
+
+  # Guardrail — ->(output) { [ok, value_or_error] }. On rejection the agent
+  # re-runs (up to guardrail_max_retries) with the reason appended.
+  guardrail: ->(out) { [out.length < 5000, 'must be under 5000 chars'] },
+  guardrail_max_retries: 3,
+
+  # Persist the result. Parent dirs are created unless create_directory: false.
+  output_file: 'out/report.md',
+  markdown: true
+)
+
+task.execute
+task.structured_output  # => { "title" => "...", "words" => 1234 }
+task.raw_result         # => the unprocessed string the agent produced
+```
+
+## 🗺️ Planning
+
+Enable `planning:` on a crew to run a planner pass before execution. The planner
+drafts a short plan for each task and folds it into the task description, giving
+the executing agent a head start:
+
+```ruby
+crew = RCrewAI::Crew.new('research_crew', planning: true)
+# Optionally use a dedicated (e.g. stronger) planner model:
+crew = RCrewAI::Crew.new('research_crew', planning: true,
+                         planning_llm: { provider: :anthropic, model: 'claude-3-opus-20240229' })
+```
+
+Planning is best-effort: if the planner errors or returns unparseable output,
+the crew runs with the original tasks unchanged.
+
+## 🏋️ Training & Testing
+
+Iterate on a crew by training it with feedback or scoring repeated runs:
+
+```ruby
+# Train: run N times, collect feedback after each run, persist to JSON.
+crew.train(n_iterations: 3, filename: 'training.json')
+
+# Provide feedback programmatically instead of prompting a human:
+crew.train(n_iterations: 3, filename: 'training.json',
+           feedback: ->(iteration, result) { "run #{iteration}: #{result[:success_rate]}%" })
+
+# Test: run N times and score each run (defaults to success_rate).
+crew.test(n_iterations: 5)
+# => { iterations: 5, scores: [...], average_score: 92.0 }
+```
+
+## 🪝 Kickoff Hooks & Batch Runs
+
+Run setup/teardown around a crew, and batch it over many inputs:
+
+```ruby
+crew.before_kickoff { |inputs| inputs.merge(started_at: Time.now) } # may transform inputs
+crew.after_kickoff  { |result| notify(result); result }            # may transform result
+
+crew.execute(inputs: { topic: 'ruby' })
+crew.last_inputs   # => the (possibly transformed) inputs the run used
+
+# Batch: run the crew once per input set, results returned in order.
+results = crew.kickoff_for_each(inputs: [
+  { topic: 'ruby' },
+  { topic: 'python' }
+])
+```
+
+## ⏱️ Rate Limiting
+
+Cap an agent's LLM calls to stay under provider limits. Calls beyond the cap
+block until the rolling 60-second window frees up:
+
+```ruby
+agent = RCrewAI::Agent.new(name: 'a', role: '...', goal: '...', max_rpm: 20)
+```
+
+The limiter (`RCrewAI::RateLimiter`) is thread-safe, so it holds under async
+execution. `max_rpm: nil` (the default) or `0` means unlimited.
+
+## 🧠 Reasoning
+
+Have an agent think through a plan before answering. The reasoning trace is
+surfaced on the result and does not pollute `task.result`:
+
+```ruby
+agent = RCrewAI::Agent.new(name: 'a', role: '...', goal: '...',
+                           reasoning: true, max_reasoning_attempts: 3)
+
+result = agent.execute_task(task)
+result[:reasoning]   # => the plan the agent drafted before answering
+result[:content]     # => the final answer
+```
+
+Off by default. If the reasoning pass keeps returning empty output past
+`max_reasoning_attempts`, the agent proceeds without a plan.
+
+## 🪟 Context Window Management
+
+Keep long tool-use loops or large injected context from overflowing the model's
+context window. When enabled, the oldest non-system messages are dropped to fit
+before each LLM call (system messages and the latest message are always kept):
+
+```ruby
+agent = RCrewAI::Agent.new(name: 'a', role: '...', goal: '...',
+                           respect_context_window: true)
+```
+
+Window sizes come from `RCrewAI::ContextWindow` (with a conservative default for
+unknown models); headroom for the response is reserved from `max_tokens`. Off by
+default.
+
+## 🖼️ Multimodal Input
+
+Pass images to a vision-capable model via task attachments. Local files are
+base64-encoded automatically; URLs pass through:
+
+```ruby
+RCrewAI.configure { |c| c.llm_provider = :openai; c.openai_model = 'gpt-4o' }
+
+task = RCrewAI::Task.new(
+  name: 'describe', description: 'What is in this chart?', agent: agent,
+  attachments: [
+    { type: :image, path: 'chart.png' },
+    { type: :image, url: 'https://example.com/photo.jpg' }
+  ]
+)
+```
+
+Supported on OpenAI and Azure; other providers raise a clear error when
+attachments are present.
+
+## 📚 Knowledge (RAG)
+
+Ground agents in your own documents. Sources are chunked, embedded, and stored
+in an in-memory vector store; the most relevant chunks are injected into each
+task's prompt automatically.
+
+```ruby
+kb = RCrewAI::Knowledge::Base.new(sources: [
+  RCrewAI::Knowledge::StringSource.new('Our refund window is 30 days.'),
+  RCrewAI::Knowledge::FileSource.new('docs/policy.txt'),
+  RCrewAI::Knowledge::PdfSource.new('handbook.pdf'),
+  RCrewAI::Knowledge::UrlSource.new('https://example.com/faq')
+])
+
+# Agent-level (role-specific) knowledge:
+support = RCrewAI::Agent.new(name: 'support', role: '...', goal: '...', knowledge: kb)
+
+# Or pass raw sources and let the agent build the base:
+support = RCrewAI::Agent.new(name: 'support', role: '...', goal: '...',
+                             knowledge_sources: [RCrewAI::Knowledge::StringSource.new('...')])
+
+# Crew-level knowledge is shared with every agent:
+crew = RCrewAI::Crew.new('support_crew', knowledge: kb)
+```
+
+Embeddings default to OpenAI's `text-embedding-3-small`; pass a custom
+`embedder:` (anything responding to `embed(texts)`) or vector store to swap the
+backend.
+
+## 🌊 Flows
+
+Beyond crews, RCrewAI has **Flows** — an event-driven workflow engine for
+orchestrating steps (and whole crews) with explicit branching and state:
+
+```ruby
+class ArticleFlow < RCrewAI::Flow
+  start :outline
+  def outline
+    state.sections = %w[intro body conclusion]
+    state.sections.length
+  end
+
+  listen :outline
+  def draft(section_count)
+    state.words = section_count * 100
+    state.words
+  end
+
+  router :draft
+  def review(words)
+    words >= 250 ? :publish : :expand
+  end
+
+  listen :publish
+  def publish = state.status = 'published'
+
+  listen :expand
+  def expand = state.status = 'needs more work'
+end
+
+flow = ArticleFlow.new
+flow.kickoff(inputs: { author: 'me' })
+flow.state.status      # => "published"
+flow.state.id          # => automatic UUID
+```
+
+- `start` / `listen` / `router` wire methods into a graph; a listener receives
+  its trigger's return value.
+- Combine triggers with `and_(:a, :b)` (all) and `or_(:a, :b)` (any).
+- **State** is a schemaless object with a UUID, seedable via `kickoff(inputs:)`.
+- **Persistence**: pass `state_store:` (`RCrewAI::Flow::FileStateStore.new(dir)`
+  or your own `#save`/`#load`) and call `flow.restore(id)` to resume.
+- Invoke a `Crew` inside any step, or pause with `human_feedback('Approve?')`.
+
 ## 💡 Examples
 
 ### Hierarchical Team with Human Oversight

data/ROADMAP.md

@@ -0,0 +1,90 @@
+# RCrewAI Roadmap
+
+This roadmap tracks feature parity between **RCrewAI** (Ruby) and the upstream
+[**crewai**](https://pypi.org/project/crewai/) Python framework.
+
+## Current status
+
+- **RCrewAI:** `0.3.0` (2026-05-12)
+- **Upstream crewai:** `1.15.x` (mid-2026)
+
+RCrewAI is a faithful port of CrewAI's **"Crews"** mental model (Agents / Tasks /
+Crew, sequential + hierarchical processes, tools, memory, human-in-the-loop). As
+of `0.3.0` the LLM plumbing is modern: native function calling across all five
+providers, a tool-schema DSL, typed streaming events, MCP client, and per-model
+pricing.
+
+Since CrewAI's `1.0`, the framework grew a second pillar (**Flows**) plus
+**Knowledge (RAG)**, **Guardrails**, **structured output**, **Planning**, and
+**Training/Testing**. RCrewAI now implements all of these — see the matrix below.
+
+**Status: complete.** All milestone issues (#5–#12) shipped in v0.4.0, and all
+backlog items (#15–#20) are done and merged to `main` (awaiting the next
+release). There is no outstanding roadmap work.
+
+## Parity matrix
+
+| Concept | crewai | RCrewAI 0.3.0 | Target |
+|---|---|---|---|
+| Agents / Tasks / Crew | ✅ | ✅ | — |
+| Sequential / hierarchical process | ✅ | ✅ | — |
+| Native function calling + tool DSL | ✅ | ✅ (0.3.0) | — |
+| Streaming events | ✅ | ✅ (0.3.0) | — |
+| MCP client | ✅ | ✅ (0.3.0) | — |
+| Per-model pricing / cost | ✅ | ✅ (0.3.0) | — |
+| Per-agent LLM override | ✅ | ✅ (#5) | ✅ done |
+| Structured output (schema) | ✅ | ✅ (#6) | ✅ done |
+| Task guardrails | ✅ | ✅ (#7) | ✅ done |
+| `output_file` / markdown | ✅ | ✅ (#8) | ✅ done |
+| Knowledge / RAG | ✅ | ✅ (#9) | ✅ done |
+| Planning | ✅ | ✅ (#10) | ✅ done |
+| Flows (`start`/`listen`/`router`) | ✅ | ✅ (#11) | ✅ done |
+| Flow state + persistence | ✅ | ✅ (#11) | ✅ done |
+| Training / Testing | ✅ | ✅ (#12) | ✅ done |
+| Reasoning, rate-limiting, batch kickoff, hooks, context window, multimodal | ✅ | ✅ (#15–#20) | ✅ done |
+
+## Milestones (highest leverage first)
+
+### 0.3.1 — Per-agent LLM override
+Let `Agent.new(llm:)` accept a provider/model, instead of only the global
+`RCrewAI.configure`. Unblocks mixed-model crews (cheap model for workers, strong
+model for the manager).
+
+### 0.4.0 — Structured output & guardrails
+Builds directly on the 0.3.0 tool-schema/JSON-schema plumbing.
+- `Task.new(output_schema:)` → validated, coerced structured result.
+- `Task.new(guardrail:)` → proc/object that validates & transforms output, with
+  bounded retries (`guardrail_max_retries`).
+- `output_file:` + `markdown:` output formatting.
+
+### 0.5.0 — Knowledge (RAG) & Planning
+- Knowledge sources: string, `.txt`, PDF (have `pdf-reader`), CSV, JSON, URL
+  (have `nokogiri`). Embeddings client + a pluggable vector store (start with an
+  in-memory / SQLite cosine store; no hard Chroma dependency).
+- Attach at agent **and** crew level.
+- `Crew.new(planning: true)` → a planner pass that drafts a step plan before
+  execution.
+
+### 0.6.0 — Flows
+The flagship. A Ruby DSL mirroring CrewAI Flows:
+- `start`, `listen`, `router` decorators/class-methods.
+- `and_` / `or_` trigger combinators.
+- Structured flow **state** (a plain struct/`Data` or dry-struct) with a UUID.
+- `@persist`-equivalent state persistence across restarts.
+- `human_feedback` pause/resume point.
+
+### 0.7.0 — Training & Testing
+- `crew.train(n_iterations:, filename:)` capturing human feedback.
+- `crew.test(n_iterations:, model:)` scoring runs.
+
+### Backlog — ✅ all complete
+
+Formerly polish items with no set version; all shipped in the `[Unreleased]`
+changes (see CHANGELOG):
+
+- [#15](https://github.com/gkosmo/rcrewAI/issues/15) — `before_kickoff` / `after_kickoff` lifecycle hooks ✅
+- [#16](https://github.com/gkosmo/rcrewAI/issues/16) — `kickoff_for_each` batch execution ✅
+- [#17](https://github.com/gkosmo/rcrewAI/issues/17) — `max_rpm` rate limiting ✅
+- [#18](https://github.com/gkosmo/rcrewAI/issues/18) — per-agent reasoning (`reasoning:`, `max_reasoning_attempts:`) ✅
+- [#19](https://github.com/gkosmo/rcrewAI/issues/19) — `respect_context_window` history trimming ✅
+- [#20](https://github.com/gkosmo/rcrewAI/issues/20) — multimodal agents (image/file inputs) ✅

data/docs/upgrading-to-0.4.md

@@ -0,0 +1,191 @@
+# Upgrading to RCrewAI 0.4
+
+RCrewAI 0.4 closes the feature-parity gap with the modern CrewAI framework. It
+adds CrewAI's second pillar — **Flows** — alongside **Knowledge (RAG)**,
+structured task output, guardrails, planning, and training/testing.
+
+**0.4 is fully backward compatible.** There are no breaking changes: existing
+0.3 code runs unchanged. Everything below is opt-in.
+
+Each new capability has a runnable example under `examples/` — see the links
+throughout.
+
+---
+
+## 1. What you must do
+
+Nothing. Upgrade the gem and your existing code keeps working:
+
+```ruby
+gem 'rcrewai', '~> 0.4'
+```
+
+---
+
+## 2. What you can do (new capabilities)
+
+### 2a. Per-agent LLM
+
+Give each agent its own provider/model instead of only the global default.
+Pass a provider symbol, an options hash, or a pre-built client:
+
+```ruby
+worker  = RCrewAI::Agent.new(name: 'worker', role: '...', goal: '...',
+                             llm: { provider: :openai, model: 'gpt-4o-mini' })
+manager = RCrewAI::Agent.new(name: 'manager', role: '...', goal: '...',
+                             llm: { provider: :anthropic, model: 'claude-3-opus-20240229' })
+```
+
+Omitting `llm:` keeps the global `RCrewAI.configure` behavior. Overrides never
+mutate the global configuration.
+
+### 2b. Structured output, guardrails, and file output
+
+Post-process a task's result after the agent produces it:
+
+```ruby
+task = RCrewAI::Task.new(
+  name: 'extract', description: '...', agent: agent,
+
+  # Validate & coerce against a JSON schema. Non-conforming output re-runs the
+  # agent with the error fed back. Parsed object on task.structured_output.
+  output_schema: { type: 'object', properties: { title: { type: 'string' } },
+                   required: ['title'] },
+
+  # Validate & transform. ->(output) { [ok, value_or_error] }. Retries up to
+  # guardrail_max_retries (default 3) with the reason fed back to the agent.
+  guardrail: ->(out) { [out.length < 5000, 'too long'] },
+
+  # Persist the result (parent dirs created unless create_directory: false).
+  output_file: 'out/report.md', markdown: true
+)
+
+task.execute
+task.structured_output  # => { "title" => "..." }
+task.raw_result         # => the unprocessed string the agent produced
+```
+
+See `examples/structured_output_example.rb`.
+
+### 2c. Knowledge (RAG)
+
+Ground agents in your own documents. Sources are chunked, embedded, and stored
+in an in-memory cosine vector store; relevant chunks are injected into each
+task's prompt automatically.
+
+```ruby
+kb = RCrewAI::Knowledge::Base.new(sources: [
+  RCrewAI::Knowledge::StringSource.new('Refunds within 30 days.'),
+  RCrewAI::Knowledge::FileSource.new('docs/policy.txt'),
+  RCrewAI::Knowledge::PdfSource.new('handbook.pdf'),
+  RCrewAI::Knowledge::UrlSource.new('https://example.com/faq')
+])
+
+# Agent-level (role-specific):
+agent = RCrewAI::Agent.new(name: 'support', role: '...', goal: '...', knowledge: kb)
+
+# Or pass raw sources and let the agent build the base:
+agent = RCrewAI::Agent.new(name: 'support', role: '...', goal: '...',
+                           knowledge_sources: [RCrewAI::Knowledge::StringSource.new('...')])
+
+# Crew-level knowledge is shared with every agent:
+crew = RCrewAI::Crew.new('support', knowledge: kb)
+```
+
+Embeddings default to OpenAI's `text-embedding-3-small`; pass a custom
+`embedder:` (anything responding to `embed(texts)`) to swap the backend.
+See `examples/knowledge_rag_example.rb`.
+
+### 2d. Planning
+
+Have a planner pass draft a per-task plan before execution:
+
+```ruby
+crew = RCrewAI::Crew.new('research_crew', planning: true)
+# Optionally use a dedicated (stronger) planner model:
+crew = RCrewAI::Crew.new('research_crew', planning: true,
+                         planning_llm: { provider: :anthropic, model: 'claude-3-opus-20240229' })
+```
+
+The plan is folded into each task's description. Planning is best-effort: a
+planner error or unparseable output leaves tasks unchanged.
+
+### 2e. Training & testing
+
+Iterate on a crew with feedback, or score repeated runs:
+
+```ruby
+# Run N times, collect feedback after each run, persist to JSON.
+crew.train(n_iterations: 3, filename: 'training.json')
+
+# Provide feedback programmatically instead of prompting a human:
+crew.train(n_iterations: 3, filename: 'training.json',
+           feedback: ->(iteration, result) { "run #{iteration}: #{result[:success_rate]}%" })
+
+# Run N times and score each run (defaults to success_rate).
+crew.test(n_iterations: 5)
+# => { iterations: 5, scores: [...], average_score: 92.0 }
+```
+
+See `examples/planning_and_training_example.rb`.
+
+### 2f. Flows
+
+Flows are an event-driven workflow engine — the biggest addition in 0.4.
+Subclass `RCrewAI::Flow` and wire methods with a class-level DSL:
+
+```ruby
+class ArticleFlow < RCrewAI::Flow
+  start :outline
+  def outline
+    state.sections = %w[intro body conclusion]
+    state.sections.length
+  end
+
+  listen :outline
+  def draft(section_count)
+    state.words = section_count * 100
+    state.words
+  end
+
+  router :draft
+  def review(words)
+    words >= 250 ? :publish : :expand
+  end
+
+  listen :publish
+  def publish = state.status = 'published'
+
+  listen :expand
+  def expand = state.status = 'needs more work'
+end
+
+flow = ArticleFlow.new
+flow.kickoff(inputs: { author: 'me' })
+flow.state.status   # => "published"
+```
+
+- `start` / `listen` / `router` wire methods into a graph; a listener receives
+  its trigger's return value, and a router's return becomes a label listeners
+  fire on.
+- Combine triggers with `and_(:a, :b)` (all) and `or_(:a, :b)` (any).
+- **State** is a schemaless object with a UUID, seedable via `kickoff(inputs:)`.
+- **Persistence**: pass `state_store:`
+  (`RCrewAI::Flow::FileStateStore.new(dir)` or your own `#save`/`#load`) and
+  call `flow.restore(id)` to resume.
+- Invoke a `Crew` inside any step, or pause with `human_feedback('Approve?')`.
+
+See `examples/flow_example.rb`.
+
+---
+
+## When to use Flows vs. Crews
+
+- **Crew** — a team of agents collaborating on a set of tasks (sequential,
+  hierarchical, or async). Reach for a crew when the work is "have these agents
+  produce these outputs."
+- **Flow** — explicit, branching orchestration with state. Reach for a flow
+  when you need conditional paths, joins, persistence/resumption, or you want to
+  coordinate multiple crews and plain Ruby steps.
+
+They compose: a Flow step can kick off a Crew.