tracebook 0.1.1 → 1.0.0

data/README.md

@@ -1,45 +1,26 @@
-# TraceBook
+# Tracebook
 
 [![Gem Version](https://img.shields.io/gem/v/tracebook.svg)](https://rubygems.org/gems/tracebook)
 [![CI](https://github.com/dpaluy/tracebook/actions/workflows/ci.yml/badge.svg)](https://github.com/dpaluy/tracebook/actions/workflows/ci.yml)
 
-> **Note:** This gem is in active development. APIs may change before 1.0 release.
+Cost tracking and review dashboard for [RubyLLM](https://github.com/crmne/ruby_llm) conversations.
 
-TraceBook is a Rails engine that ingests, redacts, and reviews LLM interactions with optional encryption. It ships with a Hotwire UI, cost tracking, rollup analytics, and adapters for popular Ruby LLM libraries.
+Tracebook is a Rails engine that sits on top of RubyLLM's `acts_as_chat` and `acts_as_message` models. It adds per-message cost calculation, chat-level review workflows, and a Hotwire-powered dashboard — without duplicating any conversation data.
 
 ## Features
 
-- **Privacy-first**: Request/response payloads are redacted (PII removal) with optional encryption at rest
-- **Cost tracking**: Automatic token usage and cost calculation per provider/model
-- **Review workflow**: Approve, flag, or reject interactions with audit trail
-- **Hierarchical sessions**: Track parent-child relationships for agent chains
-- **Analytics**: Daily rollups for reporting and cost analysis
-- **Flexible adapters**: Built-in support for multiple providers; easy to extend
-- **Production-ready**: Async job processing, export to CSV/NDJSON, filterable dashboards
+- **Cost tracking**: Per-message cost calculation based on configurable pricing rules
+- **Review workflow**: Approve or flag entire chat conversations with comments
+- **Dashboard**: Browse chats, view conversation threads, see cost/token summaries
+- **RubyLLM native**: Reads directly from your Chat and Message models — no data duplication
 
 ## Requirements
 
-- Ruby 3.2+
+- Ruby 3.4+
 - Rails 8.1+
-- ActiveJob backend (`:async` for development; Sidekiq/SolidQueue for production)
-- Database with JSONB support (PostgreSQL recommended)
-
-## Table of Contents
-
-- [Installation](#installation--setup)
-- [Configuration](#configuration)
-- [Capturing Interactions](#capturing-interactions)
-  - [Manual API](#manual-api)
-  - [Built-in Adapters](#built-in-adapters)
-- [Creating Custom Adapters](#creating-custom-adapters)
-- [Creating Custom Mappers](#creating-custom-mappers)
-- [Cost Tracking](#cost-tracking)
-- [Reviewing Data](#reviewing-data)
-- [Production Setup](#production-setup)
-  - [Securing the Dashboard](#securing-the-dashboard)
-- [Development & Testing](#development--testing)
-
-## Installation & Setup
+- [RubyLLM](https://github.com/crmne/ruby_llm) with `acts_as_chat` / `acts_as_message` models
+
+## Installation
 
 ```bash
 bundle add tracebook
@@ -47,798 +28,301 @@ bin/rails generate tracebook:install
 bin/rails db:migrate
 ```
 
-The install generator copies migrations and creates `config/initializers/tracebook.rb`.
-
-### Mount the engine
-
-Add to `config/routes.rb`:
+Mount the engine in `config/routes.rb`:
 
 ```ruby
-mount TraceBook::Engine => "/tracebook"
-```
-
-See [Securing the Dashboard](#securing-the-dashboard) for authentication options.
-
-### Optional: Configure encryption
-
-TraceBook supports ActiveRecord::Encryption for encrypting sensitive payload data at rest. This is **optional** but recommended for production environments handling sensitive data.
-
-**Step 1: Generate encryption keys**
-
-```bash
-bin/rails db:encryption:init
-```
-
-This outputs:
-
-```yaml
-active_record_encryption:
-  primary_key: [generated_key]
-  deterministic_key: [generated_key]
-  key_derivation_salt: [generated_salt]
+mount Tracebook::Engine => "/tracebook"
 ```
 
-**Step 2: Add keys to credentials**
+Seed pricing rules for common providers:
 
 ```bash
-EDITOR=vim bin/rails credentials:edit
-```
-
-```yaml
-# config/credentials.yml.enc
-active_record_encryption:
-  primary_key: <generated_key>
-  deterministic_key: <generated_key>
-  key_derivation_salt: <generated_salt>
-```
-
-**Step 3: Enable encryption in your app**
-
-Create an initializer to add encryption to the Interaction model:
-
-```ruby
-# config/initializers/tracebook_encryption.rb
-Rails.application.config.after_initialize do
-  Tracebook::Interaction.class_eval do
-    encrypts :request_payload, :response_payload
-  end
-end
+bin/rails tracebook:seed_pricing
 ```
 
-> **Note**: Enabling encryption on an existing database requires migrating existing unencrypted data. See the [Rails encryption guide](https://guides.rubyonrails.org/active_record_encryption.html) for migration strategies.
-
 ## Configuration
 
-The install generator creates `config/initializers/tracebook.rb` with sensible defaults.
-
-Available options:
-
 ```ruby
-TraceBook.configure do |config|
-  # Project identifier for filtering in the dashboard
-  config.project_name = "My App"
-
-  # Use async jobs for persistence (default: true)
-  # Set to false for tests or simple setups
-  config.persist_async = Rails.env.production?
+# config/initializers/tracebook.rb
+Tracebook.configure do |config|
+  # Class names for your RubyLLM models
+  config.chat_class = "Chat"       # default
+  config.message_class = "Message" # default
 
-  # Payload size threshold for ActiveStorage spillover (default: 64KB)
-  config.inline_payload_bytes = 64 * 1024
+  # Currency for cost calculations
+  config.default_currency = "USD"  # default
 
-  # Auto-enable adapters on boot
-  config.auto_subscribe_ruby_llm = true
-  config.auto_subscribe_active_agent = true
+  # How to display the user in the dashboard
+  config.actor_display = ->(actor) { actor.try(:name) }
 
-  # Custom PII redactors (in addition to built-in email/phone/card)
-  config.custom_redactors += [
-    ->(payload) { payload.gsub(/api_key=\w+/, "api_key=[REDACTED]") }
-  ]
+  # Items per page
+  config.per_page = 25             # default
 end
 ```
 
-Configuration is frozen after the block runs. Call `TraceBook.reset_configuration!` in tests when you need a clean slate.
-
-## Capturing Interactions
+## PII Redaction
 
-### Manual API
+Tracebook includes an opt-in PII redaction pipeline for unstructured natural language in LLM conversations. Nothing is redacted unless explicitly configured.
 
-Call `TraceBook.record!` anywhere you have access to an LLM request/response:
+### Enabling Patterns
 
 ```ruby
-TraceBook.record!(
-  provider: "openai",
-  model: "gpt-4o-mini",
-  project: "support",
-  request_payload: { messages: messages, temperature: 0.2 },
-  response_payload: response_body,
-  input_tokens: usage[:prompt_tokens],
-  output_tokens: usage[:completion_tokens],
-  latency_ms: 187,
-  status: :success,
-  tags: %w[triage priority],
-  metadata: { ticket_id: ticket.id },
-  user: current_user,
-  session_id: session_id,
-  parent_id: parent_interaction_id
-)
-```
-
-**Parameters:**
-
-- **Required:**
-  - `provider` (String) — LLM provider name (e.g., "openai", "anthropic", "ollama")
-  - `model` (String) — Model identifier (e.g., "gpt-4o", "claude-3-5-sonnet-20241022")
-
-- **Optional:**
-  - `project` (String) — Project/app name for filtering
-  - `request_payload` (Hash) — Full request sent to provider
-  - `response_payload` (Hash) — Full response from provider
-  - `request_text` (String) — Human-readable request summary
-  - `response_text` (String) — Human-readable response summary
-  - `input_tokens` (Integer) — Prompt token count
-  - `output_tokens` (Integer) — Completion token count
-  - `latency_ms` (Integer) — Request duration in milliseconds
-  - `status` (Symbol) — `:success`, `:error`, `:canceled`
-  - `error_class` (String) — Exception class name on failure
-  - `error_message` (String) — Exception message on failure
-  - `tags` (Array<String>) — Labels for filtering (e.g., ["prod", "high-priority"])
-  - `metadata` (Hash) — Custom metadata (e.g., `{ ticket_id: 123 }`)
-  - `user` (ActiveRecord object) — Associated user (polymorphic)
-  - `session_id` (String) — Session identifier for grouping related calls
-  - `parent_id` (Integer) — Parent `Interaction` ID for hierarchical chains
-
-**Return value:**
-
-```ruby
-result = TraceBook.record!(...)
-result.success?    # => true/false
-result.error       # => exception when persistence failed
-result.interaction # => AR record when persisted inline (persist_async = false)
-```
-
-When `config.persist_async = true`, the interaction is enqueued via `Tracebook::PersistInteractionJob`.
-
-### Background Jobs & Rollups
-
-**PersistInteractionJob** handles redaction, cost calculation, and writes the `Interaction` record.
-
-**DailyRollupsJob** summarizes counts, token totals, and cost into `RollupDaily` rows. Schedule it nightly per provider/model/project:
-
-```ruby
-# Example: Schedule with Sidekiq Cron or whenever
-Tracebook::DailyRollupsJob.perform_later(
-  date: Date.yesterday,
-  provider: "openai",
-  model: "gpt-4o",
-  project: nil
-)
-```
-
-Wrap this in your scheduler to cover all active provider/model/project combinations.
-
-**ExportJob** streams large CSV/NDJSON exports respecting your filters.
-
-## Built-in Adapters
-
-TraceBook ships with adapters that automatically capture LLM interactions from popular libraries. Adapters normalize provider-specific responses and call `TraceBook.record!`, so you get instrumentation without modifying application code.
+Tracebook.configure do |config|
+  # Enable individual patterns
+  config.redact :email, :phone, :ssn, :credit_card
 
-### RubyLLM Adapter
-
-The RubyLLM adapter subscribes to `ActiveSupport::Notifications` events (default: `ruby_llm.request`).
-
-**Setup:**
-
-```ruby
-# config/initializers/tracebook_adapters.rb
-TraceBook::Adapters::RubyLLM.enable!
+  # Or enable a whole group
+  config.redact :pii, :api_keys
+end
 ```
 
-**Emit events from your LLM client:**
+### Available Patterns
+
+| Pattern | Detects | Validation |
+|---------|---------|------------|
+| `email` | Email addresses | -- |
+| `phone` | Phone numbers (US format) | -- |
+| `ssn` | Social Security Numbers | SSA area-number range check |
+| `credit_card` | Credit card numbers | Luhn algorithm |
+| `openai_key` | OpenAI API keys (`sk-...`) | -- |
+| `anthropic_key` | Anthropic API keys (`sk-ant-...`) | -- |
+| `aws_key` | AWS access key IDs (`AKIA...`) | -- |
+| `stripe_key` | Stripe API keys | -- |
+| `github_token` | GitHub tokens (`ghp_`, `gho_`, etc.) | -- |
+| `ipv4` | IPv4 addresses | Octet range 0-255 |
+| `bearer_token` | Authorization bearer tokens | -- |
+| `jwt` | JSON Web Tokens | -- |
+| `private_key` | PEM-format private key blocks | -- |
+
+### Pattern Groups
+
+| Group | Patterns included |
+|-------|-------------------|
+| `pii` | `email`, `phone`, `ssn` |
+| `financial` | `credit_card` |
+| `api_keys` | `openai_key`, `anthropic_key`, `aws_key`, `stripe_key`, `github_token` |
+| `auth` | `bearer_token`, `jwt` |
+| `network` | `ipv4` |
+| `crypto` | `private_key` |
+
+### Custom Patterns
 
 ```ruby
-# Example: Wrapping an OpenAI client call
-class OpenAIService
-  def chat_completion(messages:, model: "gpt-4o", **options)
-    started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-
-    request = {
-      model: model,
-      messages: messages,
-      **options
-    }
-
-    begin
-      response = openai_client.chat(parameters: request)
-      elapsed_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000).to_i
-
-      ActiveSupport::Notifications.instrument("ruby_llm.request", {
-        provider: "openai",
-        request: request,
-        response: response,
-        meta: {
-          project: "support-chatbot",
-          tags: ["customer-support", "triage"],
-          user: current_user,
-          session_id: session.id,
-          latency_ms: elapsed_ms,
-          status: :success
-        }
-      })
-
-      response
-    rescue => e
-      elapsed_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000).to_i
-
-      ActiveSupport::Notifications.instrument("ruby_llm.request", {
-        provider: "openai",
-        request: request,
-        response: nil,
-        meta: {
-          project: "support-chatbot",
-          user: current_user,
-          session_id: session.id,
-          latency_ms: elapsed_ms,
-          status: :error,
-          error_class: e.class.name,
-          error_message: e.message
-        }
-      })
-
-      raise
-    end
-  end
-
-  private
-
-  def openai_client
-    @openai_client ||= OpenAI::Client.new(access_token: ENV['OPENAI_API_KEY'])
-  end
+Tracebook.configure do |config|
+  config.redact_pattern(
+    /policy[:\s]*\d{10}/i,
+    "[POLICY_NUMBER]",
+    name: "policy_number"
+  )
 end
 ```
 
-**Supported providers:** OpenAI, Anthropic, Ollama (built-in mappers). Other providers use the fallback mapper.
+### Custom Redactors
 
-**Custom event name:**
+Provide any callable (proc, lambda, or object responding to `call`):
 
 ```ruby
-# If your library uses a different event name
-TraceBook::Adapters::RubyLLM.enable!(instrumentation: "my_llm.complete")
+Tracebook.configure do |config|
+  config.custom_redactors << ->(text) {
+    text.gsub(/MRN-\d{8}/, "[MEDICAL_RECORD]")
+  }
+end
 ```
 
-**Disabling:**
+### Using Redaction
 
 ```ruby
-# In test environment or when switching instrumentation
-TraceBook::Adapters::RubyLLM.disable!
-```
-
-### ActiveAgent Adapter
-
-For applications using ActiveAgent (agentic frameworks), enable the bus adapter:
+# Redact text directly
+Tracebook.redact("Email user@test.com or call 555-123-4567")
+# => "Email [EMAIL] or call [PHONE]"
 
-```ruby
-# config/initializers/tracebook_adapters.rb
-TraceBook::Adapters::ActiveAgent.enable!(bus: ActiveAgent::Bus)
+# Use in your application before saving messages
+content = Tracebook.redact(user_input)
+chat.ask(content)
 ```
 
-The adapter automatically captures agent interactions including parent-child relationships for hierarchical agent chains.
+### Planned: LLM-Based Redaction
 
-**Note:** If you omit `bus:`, the adapter attempts to locate `ActiveAgent::Bus` automatically when loaded.
+For context-sensitive PII that regex can't catch (e.g. "my social is seven eight two three three three two"), a future version will support LLM-based redaction using a local model (e.g., Ollama) to detect PII in natural language before persistence.
 
-## Creating Custom Adapters
+## Tracebook Tables
 
-Adapters follow a simple pattern:
+Tracebook adds four tables — all prefixed with `tracebook_` to avoid collisions:
 
-1. Listen to whatever instrumentation your LLM client exposes (Notifications, middleware, observers, etc.)
-2. Normalize the payload using `Tracebook::Mappers.normalize` or build a `NormalizedInteraction` manually
-3. Call `TraceBook.record!(**normalized.to_h)`
+| Table | Purpose |
+|-------|---------|
+| `tracebook_message_costs` | Cost + latency per message (polymorphic join to your Message) |
+| `tracebook_chat_reviews` | Review state per chat (polymorphic join to your Chat) |
+| `tracebook_comments` | Comments on chat reviews |
+| `tracebook_pricing_rules` | Cost per token by provider/model |
 
-**Example: Custom adapter for Langchain.rb**
+Your Chat and Message tables are untouched.
 
-```ruby
-# lib/tracebook/adapters/langchain_rb.rb
-module Tracebook
-  module Adapters
-    module LangchainRb
-      extend self
-
-      def enable!
-        return if @enabled
-
-        # Hook into Langchain's middleware or callback system
-        ::Langchain::LLM::Base.after_completion do |llm, request, response, duration|
-          handle_completion(
-            provider: llm.class.provider_name,
-            request: request,
-            response: response,
-            duration_ms: (duration * 1000).to_i,
-            meta: {
-              project: "langchain-app",
-              user: Current.user,
-              session_id: Current.session_id
-            }
-          )
-        end
-
-        @enabled = true
-      end
-
-      def disable!
-        # Unhook callback
-        @enabled = false
-      end
-
-      private
-
-      def handle_completion(provider:, request:, response:, duration_ms:, meta:)
-        normalized = Tracebook::Mappers.normalize(
-          provider,
-          raw_request: request,
-          raw_response: response,
-          meta: meta.merge(latency_ms: duration_ms)
-        )
-
-        TraceBook.record!(**normalized.to_h)
-      rescue => error
-        Rails.logger.error("TraceBook LangchainRb adapter error: #{error.message}")
-      end
-    end
-  end
-end
+## Cost Calculation
 
-TraceBook = Tracebook unless defined?(TraceBook)
-```
-
-**Enable your adapter:**
+After an LLM response, call `Tracebook.calculate_cost!` to record the cost:
 
 ```ruby
-# config/initializers/tracebook_adapters.rb
-require "tracebook/adapters/langchain_rb"
-Tracebook::Adapters::LangchainRb.enable!
+Tracebook.calculate_cost!(
+  message,
+  provider: "openai",
+  model: "gpt-4o",
+  latency_ms: elapsed_ms
+)
 ```
 
-## Creating Custom Mappers
+This looks up the matching pricing rule, calculates input/output costs, and creates a `tracebook_message_costs` record joined to the message.
 
-Mappers normalize provider-specific request/response formats into TraceBook's standard schema. Create a custom mapper when the built-in ones (OpenAI, Anthropic, Ollama) don't match your provider's format.
+### Integration Example
 
-**Example: Custom mapper for Cohere**
+In a typical RubyLLM app, hook into the chat response flow:
 
 ```ruby
-# lib/tracebook/mappers/cohere.rb
-module Tracebook
-  module Mappers
-    class Cohere < Base
-      def self.normalize(raw_request:, raw_response:, meta: {})
-        new.normalize(
-          raw_request: raw_request,
-          raw_response: raw_response,
-          meta: meta
-        )
-      end
-
-      def normalize(raw_request:, raw_response:, meta: {})
-        request = symbolize(raw_request || {})
-        response = symbolize(raw_response || {})
-        meta_info = indifferent_meta(meta)
-
-        build_interaction(
-          provider: "cohere",
-          model: request[:model] || response[:model],
-          project: meta_info[:project],
-          request_payload: raw_request,
-          response_payload: raw_response,
-          request_text: request[:message] || request[:prompt],
-          response_text: extract_response_text(response),
-          input_tokens: extract_token_count(response, :prompt_tokens),
-          output_tokens: extract_token_count(response, :completion_tokens),
-          latency_ms: meta_info[:latency_ms],
-          status: meta_info[:status]&.to_sym || :success,
-          error_class: nil,
-          error_message: nil,
-          tags: Array(meta_info[:tags]).compact,
-          metadata: extract_metadata(response),
-          user: meta_info[:user],
-          parent_id: meta_info[:parent_id],
-          session_id: meta_info[:session_id]
-        )
-      end
-
-      private
-
-      def extract_response_text(response)
-        response[:text] || response.dig(:generations, 0, :text)
-      end
-
-      def extract_token_count(response, key)
-        response.dig(:meta, :billed_units, key)&.to_i
-      end
-
-      def extract_metadata(response)
-        metadata = {}
-        metadata["generation_id"] = response[:generation_id] if response[:generation_id]
-        metadata["finish_reason"] = response[:finish_reason] if response[:finish_reason]
-        compact_hash(metadata)
-      end
-    end
-  end
-end
+class ChatResponseJob < ApplicationJob
+  def perform(chat_id, content)
+    chat = Chat.find(chat_id)
 
-TraceBook = Tracebook unless defined?(TraceBook)
-```
+    chat.ask(content) do |chunk|
+      # stream chunks...
+    end
 
-**Register your mapper:**
+    # After response, calculate cost for the last assistant message
+    message = chat.messages.where(role: "assistant").last
+    model = chat.model
 
-```ruby
-# lib/tracebook/mappers.rb
-require_relative "mappers/cohere"
-
-module Tracebook
-  module Mappers
-    def normalize(provider, raw_request:, raw_response:, meta: {})
-      case provider.to_s
-      when "openai"
-        normalize_openai(raw_request, raw_response, meta)
-      when "anthropic"
-        normalize_anthropic(raw_request, raw_response, meta)
-      when "ollama"
-        normalize_ollama(raw_request, raw_response, meta)
-      when "cohere"
-        Mappers::Cohere.normalize(
-          raw_request: raw_request,
-          raw_response: raw_response,
-          meta: meta
-        )
-      else
-        fallback_normalized(provider, raw_request, raw_response, meta)
-      end
-    end
+    Tracebook.calculate_cost!(
+      message,
+      provider: model.provider,
+      model: model.model_id
+    )
   end
 end
 ```
 
-**Mapper requirements:**
-
-- Inherit from `Tracebook::Mappers::Base`
-- Implement `.normalize(raw_request:, raw_response:, meta:)`
-- Return a `Tracebook::NormalizedInteraction` instance
-- Handle missing fields gracefully (return `nil` for unavailable data)
-- Extract token counts if available, otherwise leave as `nil`
+## Pricing Rules
 
-## Cost Tracking
+Tracebook calculates costs using `PricingRule` records. Seed defaults for common providers:
 
-TraceBook automatically calculates costs based on `PricingRule` records. Create pricing rules for your providers/models:
-
-```ruby
-# db/seeds.rb or a migration
-
-# OpenAI pricing (as of 2024)
-TraceBook::PricingRule.create!(
-  provider: "openai",
-  model_pattern: "gpt-4o",
-  input_per_1k: 2.50,
-  output_per_1k: 10.00,
-  currency: "USD",
-  effective_from: Date.new(2024, 8, 6)
-)
-
-TraceBook::PricingRule.create!(
-  provider: "openai",
-  model_pattern: "gpt-4o-mini",
-  input_per_1k: 0.150,
-  output_per_1k: 0.600,
-  currency: "USD",
-  effective_from: Date.new(2024, 7, 18)
-)
-
-TraceBook::PricingRule.create!(
-  provider: "openai",
-  model_pattern: "o1",
-  input_per_1k: 15.00,
-  output_per_1k: 60.00,
-  currency: "USD",
-  effective_from: Date.new(2024, 12, 17)
-)
-
-# Anthropic pricing
-TraceBook::PricingRule.create!(
-  provider: "anthropic",
-  model_pattern: "claude-3-5-sonnet-*",
-  input_per_1k: 3.00,
-  output_per_1k: 15.00,
-  currency: "USD",
-  effective_from: Date.new(2024, 10, 22)
-)
-
-TraceBook::PricingRule.create!(
-  provider: "anthropic",
-  model_pattern: "claude-3-5-haiku-*",
-  input_per_1k: 1.00,
-  output_per_1k: 5.00,
-  currency: "USD",
-  effective_from: Date.new(2024, 11, 1)
-)
-
-# Ollama (free/local)
-TraceBook::PricingRule.create!(
-  provider: "ollama",
-  model_pattern: "*",
-  input_per_1k: 0.0,
-  output_per_1k: 0.0,
-  currency: "USD",
-  effective_from: Date.new(2024, 1, 1)
-)
+```bash
+bin/rails tracebook:seed_pricing
 ```
 
-**Glob patterns:**
-
-- `gpt-4o` — Exact match
-- `gpt-4o*` — Matches `gpt-4o`, `gpt-4o-mini`, `gpt-4o-2024-08-06`
-- `claude-3-5-*` — Matches all Claude 3.5 models
-- `*` — Matches everything (fallback rule)
-
-TraceBook uses the most specific matching rule. If multiple rules match, it prefers the most recently effective one.
-
-## Reviewing Data
-
-### Dashboard UI
-
-Visit the mount path (`/tracebook` by default) to access the dashboard.
-
-**Index screen:**
-
-- **Filters**: Provider, model, project, status, review state, tags, date range
-- **KPI tiles**: Total calls, tokens used, total cost, error rate, avg latency
-- **Interaction table**: Columns include:
-  - Timestamp
-  - Label (first 100 chars of request)
-  - User
-  - Provider/Model
-  - Tokens (input/output)
-  - Cost
-  - Duration (ms)
-  - Review state
-  - Actions (Approve/Flag/Reject, detail link)
-
-**Detail screen:**
-
-- **Header**: ID, label, user, timestamp, review state dropdown + comment form
-- **Metrics panel**: Model, duration, token breakdown, cost, HTTP status
-- **Collapsible sections**:
-  - Input (messages)
-  - Output (text + tool calls)
-  - Full JSON (request/response payloads)
-  - Error (if failed)
-- **Sidebar**: Parent/child links, tags, session breadcrumb
+This creates rules for OpenAI, Anthropic, Gemini, and Ollama models.
 
-**Keyboard shortcuts:**
-
-- `j/k` — Navigate rows
-- `a` — Approve selected
-- `f` — Flag selected
-- `r` — Reject selected
-- `?` — Show help
-
-**Bulk review:**
-
-Select multiple interactions using checkboxes, then apply a review state to all at once.
-
-### Review Workflow
-
-Interactions start in `unreviewed` state. Reviewers can transition to:
-
-- **`approved`** — Interaction is acceptable; no issues found
-- **`flagged`** — Interaction requires attention (e.g., sensitive data, unexpected behavior)
-- **`rejected`** — Interaction is problematic and should not have occurred
-
-Only `admin` users (as defined in your `authorize` proc) can change review states.
-
-## Production Setup
-
-### Securing the Dashboard
-
-The dashboard should only be accessible to trusted reviewers. Here are common approaches:
-
-**Devise with admin check:**
+### Adding Custom Rules
 
 ```ruby
-# config/routes.rb
-authenticate :user, ->(u) { u.admin? } do
-  mount TraceBook::Engine => "/tracebook"
-end
+Tracebook::PricingRule.create!(
+  provider: "xai",
+  model_glob: "grok-4-1-fast*",
+  input_cents_per_unit: 20,   # per 1k tokens
+  output_cents_per_unit: 50,
+  effective_from: Date.new(2025, 7, 1),
+  currency: "USD"
+)
 ```
 
-**Session-based constraint:**
+### Glob Patterns
 
-```ruby
-# config/routes.rb
-constraints ->(req) { req.session[:admin] } do
-  mount TraceBook::Engine => "/tracebook"
-end
-```
+- `gpt-4o` — exact match
+- `gpt-4o*` — matches `gpt-4o`, `gpt-4o-mini`, `gpt-4o-2024-08-06`
+- `claude-3-5-*` — matches all Claude 3.5 models
+- `*` — fallback for any model
 
-**HTTP Basic Auth (simple setups):**
+When multiple rules match, Tracebook prefers the most specific glob (most literal characters), then the most recent `effective_from` date.
 
-```ruby
-# config/routes.rb
-TraceBook::Engine.middleware.use Rack::Auth::Basic do |username, password|
-  ActiveSupport::SecurityUtils.secure_compare(username, ENV["TRACEBOOK_USER"]) &
-    ActiveSupport::SecurityUtils.secure_compare(password, ENV["TRACEBOOK_PASSWORD"])
-end
+## Review Workflow
 
-mount TraceBook::Engine => "/tracebook"
-```
+Reviews happen at the chat level, not per-message. In the dashboard:
 
-### Queue Adapter
+1. Open a chat to see the full conversation thread
+2. Click **Approve** or **Flag**
+3. Add comments for context
 
-Configure ActiveJob to use a production queue backend:
+Programmatic access:
 
 ```ruby
-# config/environments/production.rb
-config.active_job.queue_adapter = :sidekiq # or :solid_queue, etc.
-```
+chat = Chat.find(id)
+review = Tracebook::ChatReview.for_chat(chat)
 
-### Encryption Keys (if enabled)
-
-If you've enabled payload encryption (see [Configure encryption](#optional-configure-encryption)), store keys securely:
-
-- Use Rails encrypted credentials (`bin/rails credentials:edit`)
-- Or environment variables with a secrets manager (AWS Secrets Manager, HashiCorp Vault)
-
-### Scheduling Rollup Jobs
-
-Use a scheduler to run `DailyRollupsJob` nightly:
-
-**Sidekiq Cron:**
-
-```ruby
-# config/initializers/sidekiq_cron.rb
-Sidekiq::Cron::Job.create(
-  name: "TraceBook daily rollups - OpenAI",
-  cron: "0 2 * * *", # 2am daily
-  class: "Tracebook::DailyRollupsJob",
-  kwargs: { date: Date.yesterday, provider: "openai", model: nil, project: nil }
+review.update!(
+  review_state: :approved,
+  reviewed_by: "admin@example.com"
 )
+
+review.comments.create!(author: "admin", body: "Looks good")
 ```
 
-**Whenever:**
+### Review States
 
-```ruby
-# config/schedule.rb
-every 1.day, at: "2:00 am" do
-  runner "Tracebook::DailyRollupsJob.perform_later(date: Date.yesterday, provider: 'openai', model: nil, project: nil)"
-end
-```
+| State | Meaning |
+|-------|---------|
+| `pending` | Not yet reviewed (default) |
+| `approved` | Reviewed and accepted |
+| `flagged` | Needs attention |
 
-### Monitoring
+## Dashboard
 
-Add error tracking to catch adapter failures:
+The dashboard is available at `/tracebook/chats` (or wherever you mount the engine).
 
-```ruby
-# config/initializers/tracebook.rb
-TraceBook.configure do |config|
-  # Existing config...
+### Chat List (`/tracebook/chats`)
+- All chats with actor, model, message count, token usage, cost, review state
+- KPIs: total chats, messages, cost
 
-  # Hook into error logging
-  config.on_error = ->(error, context) do
-    Sentry.capture_exception(error, extra: context) if defined?(Sentry)
-    Rails.logger.error("TraceBook error: #{error.message} - #{context.inspect}")
-  end
-end
-```
+### Chat Detail (`/tracebook/chats/:id`)
+- Full conversation thread (user and assistant messages)
+- Per-message token counts and costs
+- Review controls (approve/flag/reset)
+- Comment thread
 
-### Database Indexes
+### Actor Display
 
-TraceBook migrations include indexes for common queries. If you add custom filters, consider additional indexes:
+By default, actors are shown as `Name` or `ClassName#id`. Customize with:
 
 ```ruby
-# db/migrate/xxx_add_custom_tracebook_indexes.rb
-class AddCustomTracebookIndexes < ActiveRecord::Migration[7.1]
-  def change
-    add_index :tracebook_interactions, [:project, :occurred_at], name: "idx_tracebook_project_time"
-    add_index :tracebook_interactions, :tags, using: :gin, name: "idx_tracebook_tags"
+config.actor_display = ->(actor) {
+  case actor
+  when User then actor.email
+  else "#{actor.class}##{actor.id}"
   end
-end
+}
 ```
 
-### Data Retention
+## Securing the Dashboard
 
-Consider archiving or deleting old interactions to manage database size:
+The engine inherits from `ActionController::Base`. Restrict access with route constraints:
 
 ```ruby
-# app/jobs/archive_old_interactions_job.rb
-class ArchiveOldInteractionsJob < ApplicationJob
-  def perform
-    cutoff = 90.days.ago
-
-    # Option 1: Delete
-    TraceBook::Interaction.where("occurred_at < ?", cutoff).delete_all
-
-    # Option 2: Export to S3 before deleting
-    interactions = TraceBook::Interaction.where("occurred_at < ?", cutoff)
-    S3Archiver.archive(interactions)
-    interactions.delete_all
-  end
+# HTTP Basic Auth
+mount Tracebook::Engine => "/tracebook",
+  constraints: ->(req) {
+    Rack::Auth::Basic::Request.new(req.env).provided? &&
+    Rack::Auth::Basic::Request.new(req.env).credentials == ["admin", ENV["TRACEBOOK_PASSWORD"]]
+  }
+
+# Devise
+authenticate :user, ->(u) { u.admin? } do
+  mount Tracebook::Engine => "/tracebook"
 end
 ```
 
 ## Development & Testing
 
-### Inside the engine repository
-
 ```bash
-cd tracebook/
-bundle install
-bundle exec rails db:migrate   # Run dummy app migrations
-bundle exec rake test          # Run full test suite
-bundle exec rubocop --fix-unsafe # Fix style issues
-```
+# Run tests
+bin/rails test
 
-### Inside a host application
-
-After updating the gem, install any new migrations:
-
-```bash
-bin/rails tracebook:install:migrations
-bin/rails db:migrate
+# Seed pricing in development
+bin/rails tracebook:seed_pricing
 ```
 
-### Testing with adapters disabled
+### Reset configuration in tests
 
 ```ruby
-# test/test_helper.rb
-class ActiveSupport::TestCase
-  setup do
-    TraceBook::Adapters::RubyLLM.disable!
-    TraceBook.reset_configuration!
-
-    TraceBook.configure do |config|
-      config.authorize = ->(*) { true }
-      config.persist_async = false # Inline for tests
-    end
-  end
-end
+setup { Tracebook.reset_configuration! }
+teardown { Tracebook.reset_configuration! }
 ```
 
-## API Documentation
-
-TraceBook uses [YARD](https://yardoc.org/) for API documentation. The full API docs are available at [rubydoc.info/gems/tracebook](https://rubydoc.info/gems/tracebook).
-
-### Generating Documentation Locally
-
-```bash
-# Install YARD
-bundle install
-
-# Generate documentation
-bundle exec rake yard
-
-# Generate and open in browser
-bundle exec rake yard:open
-
-# View documentation coverage stats
-bundle exec rake yard:stats
-```
-
-Documentation is generated in the `doc/` directory. Open `doc/index.html` in your browser to view.
-
-### Key Documentation Areas
-
-- **{Tracebook}** - Main module and `record!` method
-- **{Tracebook::Mappers}** - Provider normalization
-- **{Tracebook::Adapters::RubyLLM}** - ActiveSupport::Notifications adapter
-- **{Tracebook::Interaction}** - ActiveRecord model
-- **{Tracebook::NormalizedInteraction}** - Standard data structure
-- **{Tracebook::Result}** - Return value from `record!`
-
-## Contributing
-
-1. Fork the repo and create a topic branch
-2. Ensure `bundle exec rake test` passes
-3. Update documentation and add regression tests for new behavior
-4. Run `bundle exec rubocop -A` to fix style issues
-5. Add YARD documentation for new public methods
-6. Open a PR describing the motivation and changes
-
 ## License
 
-TraceBook is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+MIT License. See [MIT-LICENSE](MIT-LICENSE).