tracebook 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 07dade3c360434e9cfa9df70dad0b102fc6787cd6c4c808096a4cdb6edb7fb61
-  data.tar.gz: 9aa38825ebdcfd85bf4c12e4eaacbbbf4fcef5d3ed507a00cc03e12a74a0f5ed
+  metadata.gz: ae82d99eb8f5d552c59dbcb77d1a99c587ecd729a6597fea01c20411e3b866d8
+  data.tar.gz: ed9a874f85ec046d24469de3d48886ef0dee878959e964af41dd3843a9c7bd0f
 SHA512:
-  metadata.gz: eeab9cefcc540469b78420a3f582241ebb18e173e9c72d9c86e81ca99914389543e8ee515a2ce9aa055a5c46a44cd042b3b7e6f9838e2a05914b8b873126d23c
-  data.tar.gz: 79365c2393c7db94686073301913b75313ceaa253c36f2a441934ada60f0d3dca522cf481b22c74f41a7ddd35e4e1e1d6c7735778354685826ab283bf1d18ef2
+  metadata.gz: f49482e0e49ccf7d7b5dd095eaf2a2ce2899f8eb3bbbb5d6d5c4e7b1c89bfc2cdd88015a361873e8c00fa28e18cf04c29456c5942ceefd25c26799614dd1b5c3
+  data.tar.gz: 3c613e6abc82a4bd30235fbaa4d4abef6606b61ae0d913bc27ea38f514185837e41efa729ad1a73d0af367eb178fc27ce043bcd94d73c164d538ab57631c78d2

data/CHANGELOG.md

@@ -5,7 +5,21 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.1.1] - 2025-12-16
+
+### Added
+
+- **Install Generator**: `bin/rails generate tracebook:install` for simplified setup
+- **Pagination**: UI pagination for interactions list using Pagy
+
+### Fixed
+
+- **Root Route**: Added missing root route redirect to `/interactions`
+- **Payload Keys**: Fixed `build_normalized_interaction` using wrong keys for payloads
+
+### Changed
+
+- **Documentation**: Clarified that encryption is optional, added setup instructions for users who want to enable it
 
 ## [0.1.0] - 2025-11-12
 
@@ -39,5 +53,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Normalized Schema**: Consistent data structure across all providers
 - **Test Suite**: Full MiniTest coverage for models, controllers, jobs, and lib
 
-[Unreleased]: https://github.com/dpaluy/tracebook/compare/v0.1.0...HEAD
+[0.1.1]: https://github.com/dpaluy/tracebook/compare/v0.1.0...v0.1.1
 [0.1.0]: https://github.com/dpaluy/tracebook/releases/tag/v0.1.0

data/README.md

@@ -3,11 +3,13 @@
 [![Gem Version](https://img.shields.io/gem/v/tracebook.svg)](https://rubygems.org/gems/tracebook)
 [![CI](https://github.com/dpaluy/tracebook/actions/workflows/ci.yml/badge.svg)](https://github.com/dpaluy/tracebook/actions/workflows/ci.yml)
 
-TraceBook is a Rails engine that ingests, redacts, encrypts, and reviews LLM interactions. It ships with a Hotwire UI, cost tracking, rollup analytics, and adapters for popular Ruby LLM libraries.
+> **Note:** This gem is in active development. APIs may change before 1.0 release.
+
+TraceBook is a Rails engine that ingests, redacts, and reviews LLM interactions with optional encryption. It ships with a Hotwire UI, cost tracking, rollup analytics, and adapters for popular Ruby LLM libraries.
 
 ## Features
 
-- **Privacy-first**: Request/response payloads are redacted (PII removal) and encrypted at rest
+- **Privacy-first**: Request/response payloads are redacted (PII removal) with optional encryption at rest
 - **Cost tracking**: Automatic token usage and cost calculation per provider/model
 - **Review workflow**: Approve, flag, or reject interactions with audit trail
 - **Hierarchical sessions**: Track parent-child relationships for agent chains
@@ -34,78 +36,36 @@ TraceBook is a Rails engine that ingests, redacts, encrypts, and reviews LLM int
 - [Cost Tracking](#cost-tracking)
 - [Reviewing Data](#reviewing-data)
 - [Production Setup](#production-setup)
+  - [Securing the Dashboard](#securing-the-dashboard)
 - [Development & Testing](#development--testing)
 
 ## Installation & Setup
 
-### 1. Add the gem
-
-```ruby
-# Gemfile
-gem "tracebook" # or path: "../gems/tracebook" for local development
-```
-
 ```bash
-bundle install
-```
-
-### 2. Install migrations
-
-Rails engines keep migrations inside the gem. Copy them into the host app and migrate:
-
-```bash
-bin/rails railties:install:migrations FROM=tracebook
+bundle add tracebook
+bin/rails generate tracebook:install
 bin/rails db:migrate
 ```
 
-This creates the following tables:
-
-- **`tracebook_interactions`** — Main table for LLM calls
-  - Encrypted columns: `request`, `response`, `review_comment`
-  - Indexes: `provider`, `model`, `review_state`, `session_id`, `parent_id`, `occurred_at`
-  - Fields: provider, model, request (JSONB), response (JSONB), usage stats, duration, review metadata, tags, etc.
-
-- **`tracebook_pricing_rules`** — Cost per 1k tokens for provider/model patterns
-  - Fields: `provider`, `model_pattern` (glob), `input_per_1k`, `output_per_1k`, `currency`, `effective_from`
-  - Example: `provider: "openai", model_pattern: "gpt-4o*", input_per_1k: 2.50, output_per_1k: 10.00`
-
-- **`tracebook_redaction_rules`** — PII detection patterns
-  - Fields: `name`, `pattern` (regex), `detector` (class name), `replacement`, `enabled`
-  - Built-in rules: email addresses, SSNs, credit cards, phone numbers
-
-- **`tracebook_rollup_daily`** — Aggregated metrics by date/provider/model/project
-  - Composite PK: `(date, provider, model, project)`
-  - Fields: call counts, token sums, cost totals, error rates, avg duration
+The install generator copies migrations and creates `config/initializers/tracebook.rb`.
 
-Re-run `railties:install:migrations` whenever the gem adds new migrations.
+### Mount the engine
 
-### 3. Mount the engine
-
-Update `config/routes.rb` to expose the UI:
+Add to `config/routes.rb`:
 
 ```ruby
-# config/routes.rb
-Rails.application.routes.draw do
-  # Wrap with authentication/authorization
-  authenticate :user, ->(u) { u.admin? } do
-    mount TraceBook::Engine => "/tracebook"
-  end
-
-  # Or use a constraint
-  constraints -> (req) { req.session[:user_id] && User.find(req.session[:user_id])&.admin? } do
-    mount TraceBook::Engine => "/tracebook"
-  end
-end
+mount TraceBook::Engine => "/tracebook"
 ```
 
-Only trusted reviewers should access the dashboard.
+See [Securing the Dashboard](#securing-the-dashboard) for authentication options.
+
+### Optional: Configure encryption
 
-### 4. Configure encryption
+TraceBook supports ActiveRecord::Encryption for encrypting sensitive payload data at rest. This is **optional** but recommended for production environments handling sensitive data.
 
-TraceBook uses ActiveRecord::Encryption to protect sensitive data. Generate keys and configure in your credentials:
+**Step 1: Generate encryption keys**
 
 ```bash
-# Generate encryption keys
 bin/rails db:encryption:init
 ```
 
@@ -118,7 +78,7 @@ active_record_encryption:
   key_derivation_salt: [generated_salt]
 ```
 
-Add these to `config/credentials.yml.enc`:
+**Step 2: Add keys to credentials**
 
 ```bash
 EDITOR=vim bin/rails credentials:edit
@@ -132,81 +92,50 @@ active_record_encryption:
   key_derivation_salt: <generated_salt>
 ```
 
-Without these keys, TraceBook will raise an error when persisting interactions.
+**Step 3: Enable encryption in your app**
+
+Create an initializer to add encryption to the Interaction model:
+
+```ruby
+# config/initializers/tracebook_encryption.rb
+Rails.application.config.after_initialize do
+  Tracebook::Interaction.class_eval do
+    encrypts :request_payload, :response_payload
+  end
+end
+```
+
+> **Note**: Enabling encryption on an existing database requires migrating existing unencrypted data. See the [Rails encryption guide](https://guides.rubyonrails.org/active_record_encryption.html) for migration strategies.
 
 ## Configuration
 
-Create `config/initializers/tracebook.rb`:
+The install generator creates `config/initializers/tracebook.rb` with sensible defaults.
+
+Available options:
 
 ```ruby
 TraceBook.configure do |config|
-  # ========================================
-  # REQUIRED: Authorization
-  # ========================================
-  # This proc receives (user, action, resource) and must return true/false
-  # Actions: :read, :review, :export
-  config.authorize = ->(user, action, resource) do
-    case action
-    when :read
-      user&.admin? || user&.reviewer?
-    when :review
-      user&.admin?
-    when :export
-      user&.admin?
-    else
-      false
-    end
-  end
+  # Project identifier for filtering in the dashboard
+  config.project_name = "My App"
 
-  # ========================================
-  # OPTIONAL: Project & Metadata
-  # ========================================
-  # Project identifier for this application
-  config.project_name = "TraceBook Dashboard"
-
-  # ========================================
-  # OPTIONAL: Persistence
-  # ========================================
-  # Use async jobs for persistence (recommended for production)
-  # When true, TraceBook.record! enqueues PersistInteractionJob
-  # When false, writes happen inline (useful for tests)
+  # Use async jobs for persistence (default: true)
+  # Set to false for tests or simple setups
   config.persist_async = Rails.env.production?
 
-  # Payload size threshold for ActiveStorage spillover (bytes)
-  # Interactions larger than this are stored in ActiveStorage
-  config.inline_payload_bytes = 64 * 1024 # 64KB
-
-  # ========================================
-  # OPTIONAL: Cost Tracking
-  # ========================================
-  # Default currency for cost calculations
-  config.default_currency = "USD"
-
-  # ========================================
-  # OPTIONAL: Export
-  # ========================================
-  # Available export formats
-  config.export_formats = %i[csv ndjson]
-
-  # ========================================
-  # OPTIONAL: Redaction
-  # ========================================
-  # Custom PII redactors (in addition to built-in email/SSN/etc)
-  config.redactors += [
-    ->(payload) {
-      # Example: Redact API keys
-      payload.gsub(/api_key["\s]*[:=]["\s]*\K[\w-]+/, "[REDACTED]")
-    },
-    ->(payload) {
-      # Example: Redact authorization headers
-      payload.gsub(/authorization["\s]*:["\s]*\K[^"]+/, "[REDACTED]")
-    }
+  # Payload size threshold for ActiveStorage spillover (default: 64KB)
+  config.inline_payload_bytes = 64 * 1024
+
+  # Auto-enable adapters on boot
+  config.auto_subscribe_ruby_llm = true
+  config.auto_subscribe_active_agent = true
+
+  # Custom PII redactors (in addition to built-in email/phone/card)
+  config.custom_redactors += [
+    ->(payload) { payload.gsub(/api_key=\w+/, "api_key=[REDACTED]") }
   ]
 end
 ```
 
-**Important**: The `authorize` proc is mandatory. TraceBook will raise an error on boot if it's missing.
-
 Configuration is frozen after the block runs. Call `TraceBook.reset_configuration!` in tests when you need a clean slate.
 
 ## Capturing Interactions
@@ -271,7 +200,7 @@ When `config.persist_async = true`, the interaction is enqueued via `Tracebook::
 
 ### Background Jobs & Rollups
 
-**PersistInteractionJob** handles redaction, encryption, cost calculation, and writes the `Interaction` record.
+**PersistInteractionJob** handles redaction, cost calculation, and writes the `Interaction` record.
 
 **DailyRollupsJob** summarizes counts, token totals, and cost into `RollupDaily` rows. Schedule it nightly per provider/model/project:
 
@@ -705,6 +634,40 @@ Only `admin` users (as defined in your `authorize` proc) can change review state
 
 ## Production Setup
 
+### Securing the Dashboard
+
+The dashboard should only be accessible to trusted reviewers. Here are common approaches:
+
+**Devise with admin check:**
+
+```ruby
+# config/routes.rb
+authenticate :user, ->(u) { u.admin? } do
+  mount TraceBook::Engine => "/tracebook"
+end
+```
+
+**Session-based constraint:**
+
+```ruby
+# config/routes.rb
+constraints ->(req) { req.session[:admin] } do
+  mount TraceBook::Engine => "/tracebook"
+end
+```
+
+**HTTP Basic Auth (simple setups):**
+
+```ruby
+# config/routes.rb
+TraceBook::Engine.middleware.use Rack::Auth::Basic do |username, password|
+  ActiveSupport::SecurityUtils.secure_compare(username, ENV["TRACEBOOK_USER"]) &
+    ActiveSupport::SecurityUtils.secure_compare(password, ENV["TRACEBOOK_PASSWORD"])
+end
+
+mount TraceBook::Engine => "/tracebook"
+```
+
 ### Queue Adapter
 
 Configure ActiveJob to use a production queue backend:
@@ -714,9 +677,9 @@ Configure ActiveJob to use a production queue backend:
 config.active_job.queue_adapter = :sidekiq # or :solid_queue, etc.
 ```
 
-### Encryption Keys
+### Encryption Keys (if enabled)
 
-**Important:** Store encryption keys securely. Never commit them to version control.
+If you've enabled payload encryption (see [Configure encryption](#optional-configure-encryption)), store keys securely:
 
 - Use Rails encrypted credentials (`bin/rails credentials:edit`)
 - Or environment variables with a secrets manager (AWS Secrets Manager, HashiCorp Vault)
@@ -812,10 +775,10 @@ bundle exec rubocop --fix-unsafe # Fix style issues
 
 ### Inside a host application
 
-After pulling new migrations from the gem:
+After updating the gem, install any new migrations:
 
 ```bash
-bin/rails railties:install:migrations FROM=tracebook
+bin/rails tracebook:install:migrations
 bin/rails db:migrate
 ```
 

data/app/assets/stylesheets/tracebook/application.css

@@ -171,3 +171,41 @@ pre {
   border-radius: 0.5rem;
   overflow-x: auto;
 }
+
+.tb-pagination {
+  display: flex;
+  justify-content: center;
+  padding: 1rem 0;
+}
+
+.tb-pagination nav {
+  display: flex;
+  gap: 0.25rem;
+  align-items: center;
+}
+
+.tb-pagination a,
+.tb-pagination span {
+  padding: 0.5rem 0.75rem;
+  border: 1px solid var(--tb-border);
+  border-radius: 0.375rem;
+  text-decoration: none;
+  color: var(--tb-text);
+  font-size: 0.875rem;
+}
+
+.tb-pagination a:hover {
+  background: rgba(11, 95, 255, 0.08);
+  border-color: var(--tb-accent);
+}
+
+.tb-pagination .current {
+  background: var(--tb-accent);
+  color: white;
+  border-color: var(--tb-accent);
+}
+
+.tb-pagination .disabled {
+  color: var(--tb-muted);
+  cursor: not-allowed;
+}

data/app/controllers/tracebook/interactions_controller.rb

@@ -2,16 +2,16 @@
 
 module Tracebook
   class InteractionsController < ApplicationController
+    include Pagy::Method
+
     before_action :set_interaction, only: [ :show, :review ]
     helper InteractionsHelper
 
-    PER_PAGE = 100
-
     def index
       @filters = filter_params
       scope = Interaction.filtered(@filters)
       @kpis = kpis_for(scope)
-      @interactions = scope.order(created_at: :desc).limit(PER_PAGE)
+      @pagy, @interactions = pagy(scope.order(created_at: :desc), limit: Tracebook.config.per_page)
       @providers = Interaction.distinct.order(:provider).pluck(:provider)
       @models = Interaction.distinct.order(:model).pluck(:model)
       @projects = Interaction.distinct.order(:project).pluck(:project).compact

data/app/models/tracebook/interaction.rb

@@ -3,14 +3,17 @@
 module Tracebook
   # ActiveRecord model for LLM interactions.
   #
-  # Stores all data related to an LLM API call including request/response payloads
-  # (encrypted), usage metrics, cost, review state, and relationships.
+  # Stores all data related to an LLM API call including request/response payloads,
+  # usage metrics, cost, review state, and relationships.
   #
-  # ## Encrypted Fields
-  # - `request_payload` - Full request sent to provider
-  # - `response_payload` - Full response from provider
+  # ## Payload Fields
+  # - `request_payload` - Full request sent to provider (JSON)
+  # - `response_payload` - Full response from provider (JSON)
   # - `review_comment` - Reviewer's comments
   #
+  # Note: Encryption is optional. See README for instructions on enabling
+  # ActiveRecord::Encryption for these fields.
+  #
   # ## Enums
   # - `status`: `:success`, `:error`, `:canceled`
   # - `review_state`: `:pending`, `:approved`, `:flagged`, `:rejected`

data/app/views/tracebook/interactions/index.html.erb

@@ -99,6 +99,9 @@
             <% end %>
           </tbody>
         </table>
+        <nav class="tb-pagination" aria-label="Interactions pagination">
+          <%== @pagy.series_nav(anchor_string: 'data-turbo-frame="interactions_table"') %>
+        </nav>
       </turbo-frame>
     <% end %>
   </section>

data/config/routes.rb

@@ -1,4 +1,6 @@
 Tracebook::Engine.routes.draw do
+  root to: redirect("interactions")
+
   resources :interactions, only: [ :index, :show ] do
     post :review, on: :member
     post :bulk_review, on: :collection

data/lib/generators/tracebook/install/USAGE

@@ -0,0 +1,10 @@
+Description:
+    Install TraceBook by copying migrations and creating an initializer.
+
+Example:
+    bin/rails generate tracebook:install
+
+After running:
+    1. Run: bin/rails db:migrate
+    2. Mount the engine in config/routes.rb
+    3. Configure authorization in the initializer

data/lib/generators/tracebook/install/install_generator.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+require "rails/generators/base"
+
+module Tracebook
+  module Generators
+    class InstallGenerator < Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+
+      desc "Install TraceBook: copy migrations and create initializer"
+
+      def copy_migrations
+        rake "tracebook:install:migrations"
+      end
+
+      def create_initializer
+        template "initializer.rb.tt", "config/initializers/tracebook.rb"
+      end
+
+      def show_next_steps
+        say ""
+        say "TraceBook installed!", :green
+        say ""
+        say "Next steps:"
+        say "  1. Run migrations:  bin/rails db:migrate"
+        say "  2. Mount the engine in config/routes.rb:"
+        say ""
+        say "     mount TraceBook::Engine => \"/tracebook\""
+        say ""
+        say "  3. Configure authorization in config/initializers/tracebook.rb"
+        say "  4. Set up ActiveRecord encryption (see README)"
+        say ""
+      end
+    end
+  end
+end

data/lib/generators/tracebook/install/templates/initializer.rb.tt

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+TraceBook.configure do |config|
+  # config.project_name = "My App"
+
+  config.persist_async = Rails.env.production?
+
+  # config.auto_subscribe_ruby_llm = true
+  # config.auto_subscribe_active_agent = true
+end

data/lib/tracebook/config.rb

@@ -74,6 +74,10 @@ module Tracebook
     #   @return [Boolean] Auto-enable ActiveAgent adapter on boot (default: false)
     attr_accessor :auto_subscribe_active_agent
 
+    # @!attribute [rw] per_page
+    #   @return [Integer] Number of interactions per page in dashboard (default: 100)
+    attr_accessor :per_page
+
     # Creates a new configuration with default values.
     #
     # @return [Config]
@@ -87,6 +91,7 @@ module Tracebook
       @custom_redactors = []
       @auto_subscribe_ruby_llm = false
       @auto_subscribe_active_agent = false
+      @per_page = 100
     end
 
     # Returns true if configuration has been finalized.

data/lib/tracebook/version.rb

@@ -1,3 +1,3 @@
 module Tracebook
-  VERSION = "0.1.0"
+  VERSION = "0.1.1"
 end

data/lib/tracebook.rb

@@ -1,3 +1,4 @@
+require "pagy"
 require "tracebook/version"
 require "tracebook/engine"
 require "tracebook/errors"
@@ -178,8 +179,8 @@ module Tracebook
         provider: attributes.fetch(:provider),
         model: attributes.fetch(:model),
         project: attributes[:project],
-        request_payload: attributes[:request],
-        response_payload: attributes[:response],
+        request_payload: attributes[:request_payload],
+        response_payload: attributes[:response_payload],
         request_text: attributes[:request_text],
         response_text: attributes[:response_text],
         input_tokens: attributes[:input_tokens],

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: tracebook
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - dpaluy
@@ -65,6 +65,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.3'
+- !ruby/object:Gem::Dependency
+  name: pagy
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '43.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '43.0'
 description: TraceBook provides a Rails engine for capturing, storing, and reviewing
   LLM API interactions with built-in support for OpenAI, Anthropic, and Ollama. Features
   include PII redaction, cost tracking, review workflows, and export capabilities.
@@ -109,6 +123,9 @@ files:
 - db/migrate/20241112000200_create_tracebook_rollups_dailies.rb
 - db/migrate/20241112000300_create_tracebook_pricing_rules.rb
 - db/migrate/20241112000400_create_tracebook_redaction_rules.rb
+- lib/generators/tracebook/install/USAGE
+- lib/generators/tracebook/install/install_generator.rb
+- lib/generators/tracebook/install/templates/initializer.rb.tt
 - lib/tasks/tracebook_tasks.rake
 - lib/tasks/yard.rake
 - lib/tracebook.rb
@@ -158,7 +175,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.0
+rubygems_version: 3.6.9
 specification_version: 4
 summary: Rails engine for LLM interaction telemetry and review.
 test_files: []