rcrewai 0.7.0 → 0.8.0

data/docs/tutorials/agent-options.md

@@ -0,0 +1,128 @@
+---
+layout: tutorial
+title: Advanced Agent & Task Options
+description: Per-agent LLM, reasoning, rate limiting, context window, multimodal, structured output, guardrails, and lifecycle hooks
+---
+
+# Advanced Agent & Task Options
+
+A tour of the production controls added across 0.4–0.7. All are opt-in; agents
+and tasks behave as before when you don't set them.
+
+## Per-agent LLM
+
+Give each agent its own provider/model instead of only the global default:
+
+```ruby
+worker  = RCrewAI::Agent.new(name: 'worker', role: '...', goal: '...',
+                             llm: { provider: :openai, model: 'gpt-4o-mini' })
+manager = RCrewAI::Agent.new(name: 'manager', role: '...', goal: '...',
+                             llm: { provider: :anthropic, model: 'claude-3-opus-20240229' })
+```
+
+Accepts a provider symbol, an options hash, or a pre-built client. Overrides
+never mutate the global configuration.
+
+## Reasoning
+
+Have an agent draft a plan before answering. The trace is exposed on the result
+and doesn't pollute `task.result`:
+
+```ruby
+agent = RCrewAI::Agent.new(name: '...', role: '...', goal: '...',
+                           reasoning: true, max_reasoning_attempts: 3)
+result = agent.execute_task(task)
+result[:reasoning]   # the plan
+result[:content]     # the answer
+```
+
+## Rate limiting
+
+Cap an agent's LLM calls to stay under provider limits (thread-safe, holds under
+async execution):
+
+```ruby
+agent = RCrewAI::Agent.new(name: '...', role: '...', goal: '...', max_rpm: 20)
+```
+
+## Context-window management
+
+Trim history to fit the model's context window (oldest non-system messages drop
+first; system + latest always kept):
+
+```ruby
+agent = RCrewAI::Agent.new(name: '...', role: '...', goal: '...',
+                           respect_context_window: true)
+```
+
+## Multimodal input
+
+Pass images to a vision-capable model via task attachments (local files are
+base64-encoded; URLs pass through). Supported on OpenAI/Azure.
+
+```ruby
+task = RCrewAI::Task.new(
+  name: 'describe', description: 'What is in this chart?', agent: agent,
+  attachments: [
+    { type: :image, path: 'chart.png' },
+    { type: :image, url: 'https://example.com/photo.jpg' }
+  ]
+)
+```
+
+## Structured output & guardrails
+
+Validate, transform, and persist a task's result:
+
+```ruby
+task = RCrewAI::Task.new(
+  name: 'extract', description: '...', agent: agent,
+
+  output_schema: { type: 'object', properties: { title: { type: 'string' } },
+                   required: ['title'] },       # -> task.structured_output
+
+  guardrail: ->(out) { [out.length < 5000, 'too long'] },  # [ok, value_or_error]
+
+  output_file: 'out/report.md', markdown: true
+)
+task.execute
+task.structured_output   # validated object
+task.raw_result          # unprocessed string
+```
+
+Schema/guardrail failures re-run the agent with the error fed back.
+
+## Planning
+
+Run a planner pass that drafts a per-task plan before execution:
+
+```ruby
+crew = RCrewAI::Crew.new('research', planning: true)   # optional planning_llm:
+```
+
+## Lifecycle hooks & batch runs
+
+```ruby
+crew.before_kickoff { |inputs| inputs.merge(started_at: Time.now) }
+crew.after_kickoff  { |result| notify(result); result }
+
+crew.execute(inputs: { topic: 'ruby' })
+crew.last_inputs   # the resolved inputs
+
+# Run the crew once per input set:
+results = crew.kickoff_for_each(inputs: [{ topic: 'ruby' }, { topic: 'python' }])
+```
+
+## Training & testing
+
+```ruby
+crew.train(n_iterations: 3, filename: 'training.json')  # collect feedback
+crew.test(n_iterations: 5)                              # score repeated runs
+```
+
+## See also
+
+- [Flows]({{ site.baseurl }}/tutorials/flows)
+- [Knowledge (RAG)]({{ site.baseurl }}/tutorials/knowledge)
+- [Cognitive Memory]({{ site.baseurl }}/tutorials/memory)
+- [Consensual Process]({{ site.baseurl }}/tutorials/consensual-process)

data/docs/tutorials/consensual-process.md

@@ -0,0 +1,58 @@
+---
+layout: tutorial
+title: Consensual Process
+description: Multi-agent consensus — agents propose competing answers and vote to pick one
+---
+
+# Consensual Process
+
+For decisions where multiple perspectives matter, the `:consensual` process has
+several agents propose competing answers and vote to pick the best one.
+
+> **Since 0.7.0.** Earlier versions treated `:consensual` as a stub that ran
+> tasks sequentially. It now performs real consensus — if you relied on the old
+> behavior, use `process: :sequential`.
+
+## How it works
+
+For each task:
+
+1. **Propose** — up to `consensus_agents` agents (default 3, capped from the
+   crew) each produce a candidate answer.
+2. **Vote** — every participant scores each candidate 0–10 against the task's
+   description and expected output.
+3. **Pick** — the highest total score wins. Ties break toward the task's
+   assigned agent.
+
+```ruby
+crew = RCrewAI::Crew.new('panel', process: :consensual, consensus_agents: 3)
+crew.add_agent(junior)
+crew.add_agent(senior)
+crew.add_task(task)
+
+result = crew.execute   # each task goes through propose → vote → pick
+```
+
+## Cost
+
+Consensus multiplies LLM calls: roughly `N` proposals + `N × N` scoring calls per
+task, where `N` is `consensus_agents` (default 3). The cap keeps cost bounded even
+on large crews — raise or lower it to trade thoroughness for cost.
+
+## Edge cases
+
+- **One agent** → a single proposal (no meaningful vote), still a valid result.
+- **A proposer errors** → that candidate is dropped; consensus continues with the
+  rest.
+- **All proposals fail** → the task is marked failed.
+
+## When to use it
+
+Reach for `:consensual` when answer quality benefits from diversity and
+cross-checking — design decisions, judgment calls, ambiguous tasks. For
+straightforward pipelines, `:sequential` or `:hierarchical` is cheaper.
+
+## Runnable example
+
+See [`examples/consensual_process_example.rb`](https://github.com/gkosmo/rcrewAI/blob/main/examples/consensual_process_example.rb)
+— runs without an API key.

data/docs/tutorials/flows.md

@@ -0,0 +1,135 @@
+---
+layout: tutorial
+title: Flows — Event-Driven Workflows
+description: Build structured, stateful workflows with start/listen/router, combinators, and persistence
+---
+
+# Flows
+
+Crews are great for "have these agents produce these outputs." **Flows** are the
+second orchestration pillar — for workflows that need explicit branching, joins,
+persistent state, or coordination across multiple crews and plain Ruby steps.
+
+Subclass `RCrewAI::Flow` and wire methods together with a class-level DSL.
+
+## A first flow
+
+```ruby
+require 'rcrewai'
+
+class ArticleFlow < RCrewAI::Flow
+  start :outline
+  def outline
+    state.sections = %w[intro body conclusion]
+    state.sections.length          # return value is passed to listeners of :outline
+  end
+
+  listen :outline
+  def draft(section_count)
+    state.words = section_count * 100
+    state.words
+  end
+
+  router :draft
+  def review(words)
+    words >= 250 ? :publish : :expand   # a router returns a label
+  end
+
+  listen :publish
+  def publish = state.status = 'published'
+
+  listen :expand
+  def expand = state.status = 'needs more work'
+end
+
+flow = ArticleFlow.new
+flow.kickoff(inputs: { author: 'Ada' })
+flow.state.status   # => "published"
+flow.state.id       # => automatic UUID
+```
+
+## The DSL
+
+- **`start :method`** — an entry point. A flow can have several; all run first.
+- **`listen :trigger`** — runs the following method after `:trigger` completes,
+  receiving its return value.
+- **`router :trigger`** — like `listen`, but the method's return value becomes a
+  **label** that other `listen` methods can trigger on. This is how you branch.
+
+### Combining triggers
+
+```ruby
+listen and_(:fetch_a, :fetch_b)   # fires once, after BOTH complete
+def merge(...); end
+
+listen or_(:cache_hit, :cache_miss)  # fires when EITHER completes
+def proceed(...); end
+```
+
+## State
+
+`state` is a schemaless object with an automatic UUID. Read and write attributes
+directly (`state.foo = 1`), and seed initial values via `kickoff(inputs:)`:
+
+```ruby
+flow.kickoff(inputs: { topic: 'ruby', max_words: 800 })
+flow.state.topic       # => "ruby"
+```
+
+## Persistence — pause and resume
+
+Pass a `state_store:` and a flow's state is saved after each run, so you can
+restore it later by id:
+
+```ruby
+store = RCrewAI::Flow::FileStateStore.new('tmp/flows')  # or your own #save/#load
+flow  = ArticleFlow.new(state_store: store)
+flow.kickoff
+id = flow.state.id
+
+# ...later, even in a fresh process...
+resumed = ArticleFlow.new(state_store: store)
+resumed.restore(id)
+resumed.state.status   # => recovered
+```
+
+Built-in stores: `RCrewAI::Flow::MemoryStateStore` (volatile) and
+`RCrewAI::Flow::FileStateStore` (JSON on disk). Any object responding to
+`#save(id, hash)` / `#load(id)` works.
+
+## Running a crew inside a flow
+
+A flow step is just a method, so it can kick off a whole crew:
+
+```ruby
+class ResearchFlow < RCrewAI::Flow
+  def initialize(crew:, **opts)
+    super(**opts)
+    @crew = crew
+  end
+
+  start :run
+  def run
+    state.crew_result = @crew.execute(inputs: { topic: state.topic })
+  end
+end
+```
+
+## Human feedback
+
+Pause a flow for input with `human_feedback`:
+
+```ruby
+listen :draft
+def approve(_draft)
+  state.approved = human_feedback('Approve this draft?')
+end
+```
+
+Provide a handler for non-interactive runs:
+`ArticleFlow.new(feedback_handler: ->(prompt) { auto_approve(prompt) })`.
+
+## Runnable example
+
+See [`examples/flow_example.rb`](https://github.com/gkosmo/rcrewAI/blob/main/examples/flow_example.rb)
+— it runs without an API key.

data/docs/tutorials/index.md

@@ -62,6 +62,51 @@ Welcome to the RCrewAI tutorials! These step-by-step guides will take you from b
 
 ---
 
+## ✨ Capabilities (0.4 – 0.7)
+
+The features that grew RCrewAI beyond the classic crew model.
+
+### [Flows — Event-Driven Workflows]({{ site.baseurl }}/tutorials/flows)
+**Orchestrate with branching and state.** `start`/`listen`/`router`, `and_`/`or_`
+combinators, schemaless state with a UUID, persistence and resume, running crews
+as steps, and `human_feedback` pause points.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Intermediate ⭐⭐
+
+### [Knowledge (RAG)]({{ site.baseurl }}/tutorials/knowledge)
+**Ground agents in your documents.** String/file/PDF/CSV/URL sources, chunking,
+multi-provider embeddings, and agent- or crew-level attachment with automatic
+retrieval into the prompt.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Intermediate ⭐⭐
+
+### [Cognitive Memory]({{ site.baseurl }}/tutorials/memory)
+**Agents that remember.** Semantic recall (embeddings + cosine), optional SQLite
+persistence, and short-term/long-term/entity/tool memory types — with a
+zero-config default.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Intermediate ⭐⭐
+
+### [Consensual Process]({{ site.baseurl }}/tutorials/consensual-process)
+**Multi-agent voting.** Agents propose competing answers and score each other to
+pick the best; tune cost with `consensus_agents`.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Beginner ⭐
+
+### [Advanced Agent & Task Options]({{ site.baseurl }}/tutorials/agent-options)
+**Production controls.** Per-agent LLM, reasoning passes, rate limiting,
+context-window management, multimodal input, structured output, guardrails,
+planning, lifecycle hooks, and batch runs.
+
+**Prerequisites:** Getting Started tutorial  
+**Difficulty:** Intermediate ⭐⭐
+
+---
+
 ## 🏗️ Architecture & Scaling
 
 ### [Working with Multiple Crews]({{ site.baseurl }}/tutorials/multiple-crews)

data/docs/tutorials/knowledge.md

@@ -0,0 +1,80 @@
+---
+layout: tutorial
+title: Knowledge (RAG)
+description: Ground agents in your own documents with retrieval-augmented generation
+---
+
+# Knowledge (RAG)
+
+Give agents access to your own documents. Sources are chunked, embedded, and
+stored in a vector store; at execution time the most relevant chunks are
+injected into the agent's task prompt automatically.
+
+## Building a knowledge base
+
+```ruby
+require 'rcrewai'
+
+kb = RCrewAI::Knowledge::Base.new(sources: [
+  RCrewAI::Knowledge::StringSource.new('Refunds are available within 30 days.'),
+  RCrewAI::Knowledge::FileSource.new('docs/policy.txt'),
+  RCrewAI::Knowledge::PdfSource.new('handbook.pdf'),
+  RCrewAI::Knowledge::CsvSource.new('faq.csv'),
+  RCrewAI::Knowledge::UrlSource.new('https://example.com/faq')
+])
+```
+
+## Attaching knowledge
+
+**Agent-level** (role-specific):
+
+```ruby
+support = RCrewAI::Agent.new(
+  name: 'support', role: 'Support specialist', goal: 'Answer using company policy',
+  knowledge: kb
+)
+
+# Or pass raw sources and let the agent build the base:
+support = RCrewAI::Agent.new(name: 'support', role: '...', goal: '...',
+                             knowledge_sources: [RCrewAI::Knowledge::StringSource.new('...')])
+```
+
+**Crew-level** (shared with every agent):
+
+```ruby
+crew = RCrewAI::Crew.new('support_crew', knowledge: kb)
+```
+
+When a task runs, chunks relevant to the task description are retrieved and added
+to the prompt under a "Relevant Knowledge" heading.
+
+## Embeddings — pick a provider
+
+Embeddings default to OpenAI's `text-embedding-3-small`. Since 0.6.1 the embedder
+is multi-provider:
+
+```ruby
+# Local, no API key:
+embedder = RCrewAI::Knowledge::Embedder.new(provider: :ollama, model: 'nomic-embed-text')
+
+# Or :azure / :google. (:anthropic has no embeddings API and raises.)
+kb = RCrewAI::Knowledge::Base.new(sources: [...], embedder: embedder)
+```
+
+Any object responding to `embed(texts) -> [[float, ...], ...]` can be substituted.
+
+## Chunking and the vector store
+
+`Knowledge::Base.new` accepts `chunk_size:` and `overlap:` to tune how documents
+are split. The default vector store is in-memory with cosine similarity; the
+store is pluggable if you need a different backend.
+
+```ruby
+kb = RCrewAI::Knowledge::Base.new(sources: [...], chunk_size: 800, overlap: 100)
+kb.search('what is the refund window?', k: 3)   # => top-k relevant chunks
+```
+
+## Runnable example
+
+See [`examples/knowledge_rag_example.rb`](https://github.com/gkosmo/rcrewAI/blob/main/examples/knowledge_rag_example.rb)
+— it runs without an API key using a fake embedder.

data/docs/tutorials/memory.md

@@ -0,0 +1,90 @@
+---
+layout: tutorial
+title: Cognitive Memory
+description: Semantic, persistent, multi-type agent memory
+---
+
+# Cognitive Memory
+
+Agents remember what they've done and recall it on future tasks. Memory is
+**zero-config by default** (in-memory, word-overlap recall) and becomes far more
+capable when you add an embedder (semantic recall) and a store (persistence).
+
+## Zero-config
+
+Every agent gets a `Memory` scoped to itself. Nothing to set up:
+
+```ruby
+agent = RCrewAI::Agent.new(name: 'engineer', role: '...', goal: '...')
+# agent.memory records executions and recalls relevant ones automatically
+```
+
+## Semantic recall
+
+Pass an embedder and recall becomes semantic — the agent finds conceptually
+related past work even when the wording differs:
+
+```ruby
+embedder = RCrewAI::Knowledge::Embedder.new           # or provider: :ollama
+agent = RCrewAI::Agent.new(name: 'engineer', role: '...', goal: '...',
+                           memory: { embedder: embedder })
+```
+
+Recall falls back to word-overlap similarity without an embedder, and embedding
+failures fall back gracefully — **memory never breaks agent execution.**
+
+## Persistence
+
+Give memory a SQLite store and it survives restarts:
+
+```ruby
+store = RCrewAI::Memory::SqliteStore.new(path: '~/.rcrewai/memory.db')
+agent = RCrewAI::Agent.new(name: 'engineer', role: '...', goal: '...',
+                           memory: { embedder: embedder, store: store })
+```
+
+The default store is `InMemoryStore` (volatile). `SqliteStore` accepts
+`max_candidates:` (default 1000) to bound how many recent rows a search scans,
+keeping recall fast as memory grows.
+
+## Memory types
+
+The `Memory` facade exposes four underlying types:
+
+```ruby
+agent.memory.short_term   # recent executions (capped, semantic recall)
+agent.memory.long_term    # durable, deduped insights from successful runs
+agent.memory.entity       # facts about entities (people, systems) seen in work
+agent.memory.tool         # tool-call history + outcomes
+
+agent.memory.entity.entities              # => ["Alice", "AWS", ...]
+agent.memory.long_term.recall('...', limit: 3)
+```
+
+### Better entity extraction
+
+By default entities are extracted heuristically (capitalized tokens). For
+multi-word names, plug in an LLM extractor:
+
+```ruby
+extractor = RCrewAI::Memory::LlmEntityExtractor.new(agent.llm_client)
+agent = RCrewAI::Agent.new(name: '...', role: '...', goal: '...',
+                           memory: { entity_extractor: extractor })
+```
+
+## Scoping
+
+Memory is scoped per agent, so agents sharing a persistent store don't read each
+other's memories. Override with `memory: { scope: 'shared' }` for deliberate
+sharing.
+
+## The classic API still works
+
+`add_execution`, `add_tool_usage`, `relevant_executions`, `tool_usage_for`,
+`clear_short_term!`, `clear_all!`, and `stats` behave as before — the cognitive
+system is a drop-in upgrade.
+
+## Runnable example
+
+See [`examples/cognitive_memory_example.rb`](https://github.com/gkosmo/rcrewAI/blob/main/examples/cognitive_memory_example.rb)
+— semantic recall + SQLite persistence, no API key required.

data/lib/rcrewai/checkpoint/cli.rb

@@ -0,0 +1,98 @@
+# frozen_string_literal: true
+
+require 'thor'
+
+module RCrewAI
+  module Checkpoint
+    # Inspects checkpoints written by a crew run.
+    class CLI < Thor
+      DEFAULT_DIR = '.rcrewai/checkpoints'
+
+      class_option :dir, type: :string, default: DEFAULT_DIR,
+                         desc: 'Directory holding checkpoint files'
+
+      desc 'list', 'List saved checkpoint runs'
+      def list
+        ids = store.list
+        if ids.empty?
+          puts 'No checkpoints found.'
+          return
+        end
+
+        puts "Checkpoints in #{options[:dir]}:"
+        ids.sort.each do |id|
+          record = store.load(id)
+          next unless record
+
+          puts "  #{id}  #{summarize(record)}"
+        end
+      end
+
+      desc 'info RUN_ID', 'Show a checkpoint in detail'
+      def info(run_id)
+        record = store.load(run_id)
+        unless record
+          puts "No checkpoint for run id #{run_id}."
+          return
+        end
+
+        puts "run:     #{record['run_id']}"
+        puts "crew:    #{record['crew']}"
+        puts "updated: #{record['updated_at']}"
+        print_lineage(run_id, record)
+        print_tasks(record)
+      end
+
+      desc 'delete RUN_ID', 'Delete a checkpoint'
+      def delete(run_id)
+        unless store.load(run_id)
+          puts "No checkpoint for run id #{run_id}."
+          return
+        end
+
+        store.delete(run_id)
+        puts "Deleted checkpoint #{run_id}."
+      end
+
+      # Thor exits non-zero on an unhandled error rather than swallowing it.
+      def self.exit_on_failure?
+        true
+      end
+
+      private
+
+      def store
+        @store ||= FileStore.new(options[:dir])
+      end
+
+      def summarize(record)
+        tasks = record['tasks'] || {}
+        done = tasks.count { |_n, t| t['status'] == 'completed' }
+        "#{record['crew']}  #{done}/#{tasks.size} tasks"
+      end
+
+      def print_lineage(run_id, record)
+        return unless record['parent_run_id']
+
+        chain = Checkpoint.lineage(store, run_id)
+        puts "lineage: #{chain.join(' -> ')}"
+      end
+
+      def print_tasks(record)
+        tasks = record['tasks'] || {}
+        return puts 'tasks:   (none)' if tasks.empty?
+
+        puts 'tasks:'
+        tasks.each do |name, entry|
+          secs = entry['execution_time']
+          timing = secs ? format(' (%.2fs)', secs) : ''
+          puts "  #{status_mark(entry['status'])} #{name}  #{entry['status']}#{timing}"
+        end
+      end
+
+      def status_mark(status)
+        status == 'completed' ? '+' : '!'
+      end
+    end
+  end
+end