rcrewai-rails 0.6.1 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 42ec86061d4e3a880fa4c985abc0f2ec55e6c7868b95c65d62f7d6de27324042
-  data.tar.gz: f73a244bd91a867dd450f94d5fe82ab02e968f62e0a82e311a8e9e736be99e46
+  metadata.gz: e21a3a1057f9221cca3d280f2307b219d15a636995284c781404c4ed3d283fa6
+  data.tar.gz: 9b715b0cc17839eb5fb23f578905d962c4b6450d0aae2b8b1f494d549497a8b3
 SHA512:
-  metadata.gz: 366cdfeb973451b6144514e72bcfe07e3dc3269e0d9bf093f9c73949f2f660ce6f3b8f136cf3fb1af7e27f08b5210da4c785b0f3e6ac5a9827379f764733f151
-  data.tar.gz: b91f369cbdd7141f83e36a407742cc31af9ecff0a3563668ec50b91dd6689c82f2bd2cd40c3db6753ef693bd535e90782cf51cfa4afc2a074a2012683d20ed74
+  metadata.gz: 1be59f83c847e48c36b0ac61527ca9dedc61b903763be6145d6dd26354575a74d2dee3fc807a804e6f847660347c8368a28d70f33df6cb57d75d2b952b98657f
+  data.tar.gz: 0f37c9c304ef0264afeb363f07ca1b23c72045d9686869fe9325e336a79f30bc1930d08f97470a6235ad7d4dc8d1e5ebab4c78eb3de9d9ae1129ea42cb48dca8

data/CHANGELOG.md

@@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.7.0] - 2026-08-13
+
+### Added
+- Observation engine: span-tree tracing of every crew execution with per-agent,
+  per-LLM-call and per-tool-call detail (timings, token counts, cost). Includes a
+  trace waterfall at `/rcrewai/executions/:id/observation`, a cost/performance
+  dashboard at `/rcrewai/observations/costs`, live monitoring over Turbo Streams,
+  and a `rcrewai:observation:prune` rake task with retention configuration
+  (`observation_retention_days`).
+
+### Fixed
+- The install generator only wrote an initializer: it never copied a migration
+  and never mounted the engine, so following the documented install produced no
+  database tables and no reachable UI. It now creates a timestamped
+  `create_rcrewai_tables` migration, mounts the engine at `/rcrewai`, and
+  installs the fully documented initializer.
+- The install migration template was missing the `rcrewai_tools` table and the
+  tasks table's `agent` reference, so a fresh install shipped a `Tool` model with
+  no backing table. The template now matches the schema the test suite validates.
+
+### Deprecated
+- `Execution#log` and `ExecutionLog`, superseded by the observation engine. Both
+  still work and now emit a deprecation warning; scheduled for removal one minor
+  version after this release.
+
+### Requires
+- `rcrewai >= 0.7.1` for agent-level tracing. On earlier 0.7.x versions traces
+  contain only crew-level spans.
+
 ## [0.6.1] - 2026-07-07
 
 ### Fixed
@@ -123,7 +152,8 @@ existing agents, tasks, and crews build unchanged.
 ### Changed
 - Rename generators from `rcrew_a_i` to `rcrewai` namespacing.
 
-[Unreleased]: https://github.com/gkosmo/rcrewai-rails/compare/v0.6.1...HEAD
+[Unreleased]: https://github.com/gkosmo/rcrewai-rails/compare/v0.7.0...HEAD
+[0.7.0]: https://github.com/gkosmo/rcrewai-rails/compare/v0.6.1...v0.7.0
 [0.6.1]: https://github.com/gkosmo/rcrewai-rails/compare/v0.6.0...v0.6.1
 [0.6.0]: https://github.com/gkosmo/rcrewai-rails/compare/v0.5.1...v0.6.0
 [0.5.1]: https://github.com/gkosmo/rcrewai-rails/compare/v0.5.0...v0.5.1

data/README.md

@@ -16,6 +16,50 @@ Rails engine for integrating [RcrewAI](https://github.com/gkosmo/rcrewai-rails)
   - Crew: `before_kickoff`/`after_kickoff` hooks, planning, the `consensual` process, batch execution
   - Knowledge (RAG) sources and Flow persistence
 
+## Observation Engine
+
+Every crew execution is traced as a tree of spans — crew, agent, task, LLM call, and
+tool call — carrying timings, token counts, and cost.
+
+- **Trace view** at `/rcrewai/executions/:id/observation`: a waterfall of the run, with
+  prompts, tool arguments, and errors on each span.
+- **Cost and performance** at `/rcrewai/observations/costs`: spend and token totals
+  across recent executions.
+- **Live monitoring**: the trace view updates over Turbo Streams while a run is in progress.
+
+Configure it in `config/initializers/rcrewai.rb`:
+
+```ruby
+config.observation_enabled = true
+config.observation_capture_prompts = :truncated  # :none | :truncated | :full
+config.observation_prompt_max_bytes = 4_096
+config.observation_flush_mode = :batched         # :batched | :immediate
+config.observation_retention_days = 30
+```
+
+Prompt text is truncated by default: full prompts can be large and may contain personal
+data. Set `:full` only when you need lossless replay.
+
+Prune old spans with the bundled rake task:
+
+```bash
+rake rcrewai:observation:prune        # uses observation_retention_days
+rake rcrewai:observation:prune DAYS=7
+```
+
+### Limitations
+
+Token and cost data depend on rcrewai's **streaming** execution path. `Usage` events are
+only emitted when an agent runs via `ToolRunner`, which passes a `stream:` to the LLM
+client — not via `LegacyReactRunner`, which does not. `ToolRunner` is selected when the
+tools have JSON schemas **and** the LLM client reports `supports_native_tools?`. OpenAI,
+Anthropic, and Google all report true, so cost capture works normally with them. With a
+provider or configuration that falls back to `LegacyReactRunner` (for example Ollama
+without native tools), **the cost dashboard will be empty rather than showing an error**.
+
+Agent-level tracing requires **rcrewai >= 0.7.1**, the version that threads the event
+stream down to agent execution. On earlier versions traces contain only crew-level spans.
+
 ## Installation
 
 Add this line to your application's Gemfile:

data/app/controllers/rcrewai/rails/observations_controller.rb

@@ -0,0 +1,27 @@
+module RcrewAI
+  module Rails
+    class ObservationsController < ApplicationController
+      # Hard ceiling on how deep the span partial will recurse. Real traces
+      # are a handful of levels deep; this only exists so a malformed tree
+      # cannot recurse until the request dies.
+      MAX_SPAN_DEPTH = 50
+
+      def show
+        @execution = Execution.find(params[:execution_id])
+        # Load the whole tree in one query and nest in memory — a recursive
+        # per-node query would be N+1 on deep traces.
+        @spans = @execution.spans.ordered.to_a
+        @children = @spans.group_by(&:parent_span_id)
+        @roots = @children[nil] || []
+      end
+
+      def costs
+        @executions = Execution.where.not(total_cost_usd: nil)
+                               .order(created_at: :desc)
+                               .limit(100)
+        @total_cost = @executions.sum(&:total_cost_usd)
+        @total_tokens = @executions.sum { |e| e.total_tokens.to_i }
+      end
+    end
+  end
+end

data/app/jobs/rcrewai/rails/crew_execution_job.rb

@@ -11,6 +11,7 @@ module RcrewAI
           inputs: inputs,
           batch_id: batch_id
         )
+        collector = nil
 
         begin
           execution.start!
@@ -18,7 +19,16 @@ module RcrewAI
 
           rcrew = crew.to_rcrew
 
-          result = rcrew.execute(stream: stream_sink_for(execution))
+          collector = collector_for(execution)
+          collector&.start_crew_span(crew_name: crew.name)
+
+          result = rcrew.execute(stream: collector)
+
+          # Close agent spans as successful before finish!, which treats
+          # anything still open as an aborted run.
+          collector&.finish_open_agent_spans(status: "ok")
+          collector&.finish_crew_span(status: "ok")
+          collector&.finish!
 
           execution.complete!(result)
           execution.log("info", "Crew execution completed", { result: result })
@@ -27,6 +37,7 @@ module RcrewAI
 
           result
         rescue => e
+          collector&.finish!
           execution.fail!(e)
           execution.log("error", "Crew execution failed", {
             error: e.message,
@@ -39,34 +50,12 @@ module RcrewAI
 
       private
 
-      # Build a stream sink that translates rcrewai events into ExecutionLog rows.
-      # Note: the gem currently builds the sink but does not yet thread it down
-      # to per-agent execution, so this is wired up for forward-compatibility.
-      def stream_sink_for(execution)
-        lambda do |event|
-          case event
-          when RCrewAI::Events::IterationStart
-            execution.log("debug", "Iteration #{event.iteration_index} start", { agent: event.agent })
-          when RCrewAI::Events::IterationEnd
-            execution.log("debug", "Iteration end", { agent: event.agent, finish_reason: event.finish_reason })
-          when RCrewAI::Events::ToolCallStart
-            execution.log("info", "Tool call: #{event.tool}", { args: event.args, agent: event.agent })
-          when RCrewAI::Events::ToolCallResult
-            execution.log("info", "Tool result: #{event.tool}", { duration_ms: event.duration_ms, agent: event.agent })
-          when RCrewAI::Events::ToolCallError
-            execution.log("error", "Tool error: #{event.tool}", { error: event.error, agent: event.agent })
-          when RCrewAI::Events::Usage
-            execution.log("debug", "Usage", {
-              prompt_tokens: event.prompt_tokens,
-              completion_tokens: event.completion_tokens,
-              total_tokens: event.total_tokens,
-              cost_usd: event.cost_usd,
-              agent: event.agent
-            })
-          when RCrewAI::Events::Error
-            execution.log("error", "Crew error", { error: event.error, agent: event.agent })
-          end
-        end
+      # Translates rcrewai events into the span tree. Returns nil when
+      # observation is disabled so no sink is attached at all.
+      def collector_for(execution)
+        return nil unless RcrewAI::Rails.config.observation_enabled
+
+        RcrewAI::Rails::Observation::Collector.new(execution: execution)
       end
 
       def notify_completion(crew, execution, result)

data/app/models/rcrewai/rails/execution.rb

@@ -5,6 +5,7 @@ module RcrewAI
       
       belongs_to :crew
       has_many :execution_logs, dependent: :destroy
+      has_many :spans, dependent: :destroy
 
       validates :status, inclusion: { in: %w[pending running completed failed cancelled] }
 
@@ -74,7 +75,12 @@ module RcrewAI
         %w[completed failed cancelled].include?(status)
       end
 
+      # Deprecated: superseded by the observation engine's span tree.
+      # Scheduled for removal one minor version after the engine ships.
       def log(level, message, details = {})
+        RcrewAI::Rails.deprecator_warn(
+          "Execution#log and ExecutionLog are deprecated; use the observation engine (Execution#spans)."
+        )
         execution_logs.create!(
           level: level,
           message: message,

data/app/models/rcrewai/rails/span.rb

@@ -0,0 +1,57 @@
+module RcrewAI
+  module Rails
+    class Span < ApplicationRecord
+      self.table_name = "rcrewai_spans"
+
+      KINDS    = %w[crew agent task llm_call tool_call].freeze
+      STATUSES = %w[running ok error].freeze
+
+      belongs_to :execution
+      belongs_to :parent, class_name: "RcrewAI::Rails::Span",
+                          foreign_key: :parent_span_id, optional: true
+      has_many :children, class_name: "RcrewAI::Rails::Span",
+                          foreign_key: :parent_span_id, dependent: :destroy
+      has_many :span_events, dependent: :destroy
+
+      validates :kind, inclusion: { in: KINDS }
+      validates :status, inclusion: { in: STATUSES }
+      validates :name, :trace_id, :started_at, :sequence, presence: true
+
+      scope :roots, -> { where(parent_span_id: nil).order(:sequence) }
+      scope :ordered, -> { order(:sequence) }
+      scope :errored, -> { where(status: "error") }
+      scope :running, -> { where(status: "running") }
+      scope :llm_calls, -> { where(kind: "llm_call") }
+      scope :tool_calls, -> { where(kind: "tool_call") }
+
+      def attributes_hash
+        raw = self[:attributes_json]
+        return {} if raw.blank?
+
+        JSON.parse(raw)
+      rescue JSON::ParserError
+        {}
+      end
+
+      def attributes_hash=(hash)
+        self[:attributes_json] = hash.nil? ? nil : JSON.generate(hash)
+      end
+
+      def finish!(status: "ok", ended_at: Time.current)
+        update!(
+          status: status,
+          ended_at: ended_at,
+          duration_ms: ((ended_at - started_at) * 1000).round
+        )
+      end
+
+      def running?
+        status == "running"
+      end
+
+      def errored?
+        status == "error"
+      end
+    end
+  end
+end

data/app/models/rcrewai/rails/span_event.rb

@@ -0,0 +1,19 @@
+module RcrewAI
+  module Rails
+    class SpanEvent < ApplicationRecord
+      self.table_name = "rcrewai_span_events"
+
+      LEVELS = %w[debug info warn error].freeze
+
+      belongs_to :span
+
+      validates :level, inclusion: { in: LEVELS }
+      validates :name, presence: true
+
+      serialize :details, coder: JSON
+
+      scope :errors, -> { where(level: "error") }
+      scope :recent, -> { order(timestamp: :desc) }
+    end
+  end
+end

data/app/views/rcrewai/rails/observations/_span.html.erb

@@ -0,0 +1,51 @@
+<%#
+  Renders one span and, recursively, its children.
+
+  `children` is the parent_span_id => [spans] map built once by the
+  controller. Children are read from that map, never via `span.children`,
+  which would issue a query per node and turn a deep trace into an N+1.
+
+  A malformed tree (a parent/child cycle, e.g. a bad backfill) would
+  otherwise recurse until the request dies, so recursion is bounded by
+  MAX_SPAN_DEPTH and by `seen`, which drops any span already on the path.
+%>
+<% depth = local_assigns.fetch(:depth, 0) %>
+<% children = local_assigns.fetch(:children, {}) %>
+<% seen = local_assigns.fetch(:seen, []) %>
+<% max_depth = RcrewAI::Rails::ObservationsController::MAX_SPAN_DEPTH %>
+
+<div class="span-row span-<%= span.status %> span-kind-<%= span.kind %>" id="span-<%= span.id %>">
+  <div class="span-line" style="padding-left: <%= depth * 20 %>px">
+    <span class="span-kind"><%= span.kind %></span>
+    <span class="span-name"><%= span.name %></span>
+    <span class="span-status status-<%= span.status %>"><%= span.status %></span>
+    <% if span.duration_ms %>
+      <span class="span-duration"><%= span.duration_ms %>ms</span>
+    <% end %>
+    <% if span.total_tokens.to_i.positive? %>
+      <span class="span-tokens"><%= span.total_tokens %> tokens</span>
+    <% end %>
+    <% if span.cost_usd.present? %>
+      <span class="span-cost">$<%= span.cost_usd %></span>
+    <% end %>
+  </div>
+
+  <% attrs = span.attributes_hash %>
+  <% if attrs.present? %>
+    <details class="span-attributes" style="margin-left: <%= (depth * 20) + 20 %>px">
+      <summary>Details</summary>
+      <pre class="code-block"><%= JSON.pretty_generate(attrs) %></pre>
+    </details>
+  <% end %>
+</div>
+
+<% if depth < max_depth %>
+  <% (children[span.id] || []).each do |child| %>
+    <% next if seen.include?(child.id) %>
+    <%= render "rcrewai/rails/observations/span",
+               span: child,
+               children: children,
+               depth: depth + 1,
+               seen: seen + [span.id] %>
+  <% end %>
+<% end %>

data/app/views/rcrewai/rails/observations/costs.html.erb

@@ -0,0 +1,52 @@
+<div class="page-header">
+  <h1>Cost &amp; Performance</h1>
+</div>
+
+<div class="metadata-grid">
+  <div class="meta-item">
+    <strong>Total Cost:</strong> $<%= @total_cost %>
+  </div>
+  <div class="meta-item">
+    <strong>Total Tokens:</strong> <%= @total_tokens %>
+  </div>
+  <div class="meta-item">
+    <strong>Executions:</strong> <%= @executions.size %>
+  </div>
+</div>
+
+<div class="section">
+  <h2>Recent Executions</h2>
+
+  <% if @executions.empty? %>
+    <p class="empty-state">No executions with recorded cost yet.</p>
+  <% else %>
+    <table class="table">
+      <thead>
+        <tr>
+          <th>Execution</th>
+          <th>Crew</th>
+          <th>Status</th>
+          <th>Spans</th>
+          <th>Errors</th>
+          <th>Tokens</th>
+          <th>Cost</th>
+          <th></th>
+        </tr>
+      </thead>
+      <tbody>
+        <% @executions.each do |execution| %>
+          <tr>
+            <td><%= link_to "##{execution.id}", execution_path(execution) %></td>
+            <td><%= execution.crew&.name %></td>
+            <td><span class="status-<%= execution.status %>"><%= execution.status.humanize %></span></td>
+            <td><%= execution.span_count.to_i %></td>
+            <td><%= execution.error_count.to_i %></td>
+            <td><%= execution.total_tokens.to_i %></td>
+            <td>$<%= execution.total_cost_usd %></td>
+            <td><%= link_to "Trace", execution_observation_path(execution), class: "btn btn-sm" %></td>
+          </tr>
+        <% end %>
+      </tbody>
+    </table>
+  <% end %>
+</div>

data/app/views/rcrewai/rails/observations/show.html.erb

@@ -0,0 +1,47 @@
+<div class="page-header">
+  <h1>Trace for Execution #<%= @execution.id %></h1>
+  <div class="actions">
+    <%= link_to "Back to Execution", execution_path(@execution), class: "btn" %>
+    <%= link_to "Costs", observation_costs_path, class: "btn" %>
+  </div>
+</div>
+
+<div class="execution-details">
+  <div class="status-banner status-<%= @execution.status %>">
+    <strong>Status:</strong> <%= @execution.status.humanize %>
+  </div>
+
+  <div class="metadata-grid">
+    <div class="meta-item">
+      <strong>Spans:</strong> <%= @execution.span_count.to_i %>
+    </div>
+    <div class="meta-item">
+      <strong>Errors:</strong> <%= @execution.error_count.to_i %>
+    </div>
+    <div class="meta-item">
+      <strong>Tokens:</strong> <%= @execution.total_tokens.to_i %>
+    </div>
+    <div class="meta-item">
+      <strong>Cost:</strong> <%= @execution.total_cost_usd.present? ? "$#{@execution.total_cost_usd}" : "N/A" %>
+    </div>
+  </div>
+</div>
+
+<div class="section">
+  <h2>Waterfall</h2>
+
+  <div class="span-tree" id="execution-<%= @execution.id %>-spans">
+    <% if @roots.empty? %>
+      <p class="empty-state">No spans recorded for this execution.</p>
+    <% else %>
+      <% @roots.each do |span| %>
+        <%= render "rcrewai/rails/observations/span",
+                   span: span, children: @children, depth: 0, seen: [] %>
+      <% end %>
+    <% end %>
+  </div>
+</div>
+
+<% if defined?(turbo_stream_from) && @execution.running? %>
+  <%= turbo_stream_from "rcrewai_execution_#{@execution.id}" %>
+<% end %>

data/config/routes.rb

@@ -16,6 +16,12 @@ RcrewAI::Rails::Engine.routes.draw do
     end
   end
 
+  resources :executions, only: [] do
+    resource :observation, only: [:show], controller: "observations"
+  end
+
+  get "observations/costs", to: "observations#costs", as: :observation_costs
+
   resources :agents do
     resources :tools
   end

data/db/migrate/010_create_rcrewai_spans.rb

@@ -0,0 +1,42 @@
+class CreateRcrewaiSpans < ActiveRecord::Migration[7.0]
+  def change
+    create_table :rcrewai_spans do |t|
+      t.references :execution, null: false, foreign_key: { to_table: :rcrewai_executions }
+      t.bigint :parent_span_id
+      t.string :trace_id, null: false
+      t.string :kind, null: false
+      t.string :name, null: false
+      t.string :status, null: false, default: "running"
+      t.datetime :started_at, null: false
+      t.datetime :ended_at
+      t.integer :duration_ms
+      t.integer :prompt_tokens
+      t.integer :completion_tokens
+      t.integer :total_tokens
+      t.decimal :cost_usd, precision: 12, scale: 6
+      t.text :attributes_json
+      t.integer :sequence, null: false
+
+      t.timestamps
+    end
+
+    add_index :rcrewai_spans, :parent_span_id
+    add_index :rcrewai_spans, :trace_id
+    add_index :rcrewai_spans, :kind
+    add_index :rcrewai_spans, :status
+    add_index :rcrewai_spans, %i[execution_id sequence]
+
+    create_table :rcrewai_span_events do |t|
+      t.references :span, null: false, foreign_key: { to_table: :rcrewai_spans }
+      t.string :level, null: false, default: "info"
+      t.string :name, null: false
+      t.text :details
+      t.datetime :timestamp, null: false
+
+      t.timestamps
+    end
+
+    add_index :rcrewai_span_events, :level
+    add_index :rcrewai_span_events, :timestamp
+  end
+end

data/db/migrate/011_add_observation_rollups_to_rcrewai_executions.rb

@@ -0,0 +1,8 @@
+class AddObservationRollupsToRcrewaiExecutions < ActiveRecord::Migration[7.0]
+  def change
+    add_column :rcrewai_executions, :total_cost_usd, :decimal, precision: 12, scale: 6
+    add_column :rcrewai_executions, :total_tokens, :integer
+    add_column :rcrewai_executions, :span_count, :integer, default: 0, null: false
+    add_column :rcrewai_executions, :error_count, :integer, default: 0, null: false
+  end
+end