railwatch 0.1.2 → 0.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 50160a5f39e8b422581a3b6e6cf0b07eddc3565cfc17a8ba522bc49e15cd93df
-  data.tar.gz: 4f2e2fdac0c85df89a9d595fb33f02ebcf5c02a6f88ed81cb6dc19a163a2b90d
+  metadata.gz: 0b78b98999447f64098f912cdfdcbb6202c9331ac4e07988fb350080d084f79c
+  data.tar.gz: e230f37a0a728c381fc4b84849bc0db762ad14e2b9c37aa5802850b97cba7271
 SHA512:
-  metadata.gz: 584a8545680ad9c2d24ef05f6fcfd331a933cb87a310859a6c6e7df6c447b1d1f1d1ace62ccf11b899ae9bbcf0255d3b397562e57ba402e2410ba5790a99b144
-  data.tar.gz: 0fd7c798ae6d8b745e5e19ed9a894331b09eb6b15a179d9d6cda6d6bbb4c1d8a6b8bf15eefecdb24db57fd2b956b1eabba1085e0aba7fa074810aa2d5f41374c
+  metadata.gz: 27c7659440a7e1592b99d10ad2a409c711a2b5a44cafee9a4707ff8672e0952fcf6819ad5b98c96f4e9d25ae6e1c5b86193ebb70a74b52904f8aee808977c72d
+  data.tar.gz: c1101b7933f5703f4dfd7469b07bf9ac3f39cfaee7bd2a9d451e30126921c33a0326c5976d8605e7dce39f13b10b8d65fd8052007051775bc9e46f0b5cbb6368

data/CHANGELOG.md

@@ -1,5 +1,52 @@
 # Changelog
 
+## 0.1.4 (2026-09-15)
+
+- `llm_call` records what the call carried and how it was configured, not
+  just what it cost. `finish_reason` shows when an answer was cut off
+  (`max_tokens`) or filtered, which previously read exactly like a complete
+  one. `attachments` and `attachment_types` show that a call carried two
+  images and a PDF, so a document read is no longer indistinguishable from
+  an expensive prompt -- on a document-reading call the attachments are most
+  of the input tokens. `params` holds the settings that produced the answer
+  (temperature, max_output_tokens, tool_choice, thinking, caching, whether a
+  schema was used, and the per-operation ones) so a surprising result can be
+  reproduced. `tools` names what the model could reach; `tool_call_id` joins
+  a tool call to the turn that asked for it.
+- `provider_request_id` is read from the response headers (`request-id`,
+  `x-request-id`, `x-amzn-requestid`). It is the only key that joins a
+  Railwatch record to the provider's own record of the same call, and it is
+  what a provider support ticket asks for.
+- `cost_reported` distinguishes a price the provider stated from one
+  estimated against the model registry.
+- `provider_options` is filtered twice before it is stored: through the
+  app's own parameter filter, and again against the credential-name matcher
+  that catches `X-Api-Key` on a header. The default parameter filter is
+  password-shaped, so an `api_key` passed per call went through it
+  untouched. Attachment filenames stay behind `capture_llm_content`, since
+  a filename is business data rather than metadata.
+
+## 0.1.3 (2026-09-15)
+
+- LLM calls are recorded from RubyLLM's own instrumentation. Every model
+  call it emits -- `chat`, `compaction`, `embedding`, `image`, `speech`,
+  `transcription`, `moderation`, `rerank`, `ocr` -- plus each tool
+  invocation becomes one
+  `llm_call` child record on the request, job, or command that made it,
+  carrying provider, model, duration, token counts per bucket, and cost.
+  Nothing is patched: RubyLLM publishes ActiveSupport::Notifications events
+  and Railwatch subscribes to them like any Rails event.
+- Both RubyLLM generations are read from the same subscriber. 1.16 puts
+  token counts on the event as scalars and reports no cost; 2.0 sends its
+  `Tokens` and `Cost` objects, and stamps `workflow_id` and step identity on
+  every event inside `RubyLLM.workflow`, which is recorded so an agent run
+  can be reassembled from its steps. A 1.16 app has no cost rather than a
+  cost of zero, and a model the registry cannot price is unpriced, not free.
+- `capture_llm_content` (default off, `RAILWATCH_CAPTURE_LLM_CONTENT`)
+  records the last user turn and the reply, capped at 4 KiB of bytes each. Token
+  counts, model, and cost are always captured; prompts are not, because
+  they are whatever the app sent a provider.
+
 ## 0.1.2 (2026-09-14)
 
 - A failed job's exception is reported once. Solid Queue re-raises it out

data/docs/records.md

@@ -502,6 +502,78 @@ never double-recorded.
 | `response_body` | First 4 KiB of the response body, but only when `config.capture_response_body_on_error` is on (off by default) *and* the response was an error. A body that parses as a JSON object is run through the same parameter filter as request params and re-serialized; anything else is stored as it arrived. nil in every other case — including a connection failure, where there is no response (on the Net::HTTP path a body is read only if Net::HTTP already buffered it, so a response being streamed through `read_body` is never consumed; on the Faraday path the body is taken only once a status came back, so an outgoing request payload can never be filed as a response). |
 | `source` | App-code call site (Net::HTTP path only). |
 
+### `llm_call`
+
+Every RubyLLM model call and tool invocation, from
+`lib/railwatch/subscribers/llm.rb`. RubyLLM publishes its own
+`ActiveSupport::Notifications` events, so nothing is patched and RubyLLM is
+not a dependency — an app without it never emits these. Requires RubyLLM
+1.16 or later, which is where its instrumentation landed.
+
+The model call also appears as an `outgoing_request`, since it is an HTTP
+call like any other. The two are different grains on purpose: the
+`outgoing_request` is the HTTP truth, the `llm_call` is what it cost. That
+difference is useful: RubyLLM retries through Faraday, so one `llm_call`
+with several `outgoing_request` rows against it in the same execution is a
+call that was retried. Over a window, `outgoing_requests - llm_calls` to the
+same provider host is the number of *extra attempts*, not a rate -- the
+share of calls that were retried needs counting the calls with more than one
+request against them, which the execution id supports.
+
+**Token counts and cost differ by RubyLLM version.** 1.16 reports token
+counts and no cost at all. 2.0 reports both, from its usage ledger, and
+adds the `workflow_*` fields. `cost_nanos` is null rather than zero
+whenever RubyLLM reported no cost or the model registry could not price
+it — an unpriced call is not a free one.
+
+**Concurrent tool calls are not recorded.** RubyLLM's opt-in
+`tool_concurrency` (`:threads` or `:fibers`) runs each tool in a fresh
+thread or fiber. `Railwatch::Current` is backed by
+`ActiveSupport::IsolatedExecutionState`, which a new thread does not
+inherit, so the `tool_call.ruby_llm` event fires with no execution to
+attach to and the record is dropped rather than misattributed. This
+affects every Railwatch subscriber in an app-spawned thread, not just this
+one. Tool concurrency is off by default; with it off, tool calls are
+recorded normally. The model calls themselves are unaffected either way,
+so cost is always complete.
+
+| Field | Meaning |
+|---|---|
+| `group` | Hash of provider + model + operation, or of `"tool"` + tool name. |
+| `operation` | `"chat"`, `"compaction"`, `"embedding"`, `"image"`, `"speech"`, `"transcription"`, `"moderation"`, `"rerank"`, `"ocr"`, or `"tool"`. |
+| `provider` | Provider slug, e.g. `"anthropic"`. |
+| `model` | Model the call was made with. Empty for a provider that selects its own (moderation). |
+| `response_model` | Model the provider says answered, which can differ from the one asked for. |
+| `tool_name` | Tool name, for `operation: "tool"`. |
+| `duration` | Microseconds. |
+| `status` | `"ok"`, or `"failed"` if the call raised. |
+| `error` | `"Class: message"`, truncated to 255 chars, if the call raised. |
+| `streaming` | Whether the call was streamed. |
+| `message_count` | Conversation length at the time of the call. |
+| `tool_count` | Number of tools the model was offered. |
+| `input_tokens` | Standard (non-cached) input tokens. |
+| `output_tokens` | Billable output tokens. |
+| `cache_read_tokens` | Tokens served from the provider's prompt cache. |
+| `cache_write_tokens` | Tokens written to the provider's prompt cache. |
+| `thinking_tokens` | Reasoning tokens, where the provider reports them separately. |
+| `cost_nanos` | Cost in billionths of a US dollar. Null when unpriced — see above. Nanodollars because a cheap call is well under a microdollar and floats do not sum to an invoice. |
+| `workflow_id` | `RubyLLM.workflow` identifier (2.0+). Null outside a workflow. |
+| `workflow_name` | Workflow name (2.0+). |
+| `workflow_step_id` | Step identifier within the workflow (2.0+). |
+| `workflow_step_name` | Step name (2.0+). |
+| `workflow_step_parent_id` | Enclosing step, for nested steps — what reconstructs the tree (2.0+). |
+| `finish_reason` | Why the model stopped: `stop`, `max_tokens`, `tool_calls`, `content_filter`, or whatever the provider spelled it. `max_tokens` means the answer was cut off -- without this a truncated extraction reads exactly like a complete one. |
+| `provider_request_id` | The provider's own id for the request, read from the response headers (`request-id`, `x-request-id`, `x-amzn-requestid`). The only key that joins this record to the provider's side of it, and what a support ticket asks for. |
+| `tools` | Comma-separated names of the tools the model could reach, first 50. `tool_count` says how many; retracing needs which. |
+| `cost_reported` | Whether the provider priced the call itself, or the amount is an estimate from the model registry. Null on gems or operations that report no cost. |
+| `attachments` | How many files the last user turn carried. Absent when it carried none. Only the last turn is measured: earlier turns were counted by the calls that sent them. |
+| `attachment_types` | What they were, by category and count, e.g. `imagex2,pdf`. Categories are RubyLLM's: image, pdf, audio, video, text, document, unknown. On a document-reading call the attachments are most of the input tokens, so without this an expensive scan is indistinguishable from an expensive prompt. |
+| `attachment_names` | Filenames, only when `config.capture_llm_content` is on. A filename like `ACME_invoice_88231.pdf` is business data, not metadata, so it follows the same switch as prompts. |
+| `params` | JSON of the settings that produced the answer, so a surprising one can be reproduced: `temperature`, `max_output_tokens`, `tool_choice`, `tool_call_limit`, `thinking`, `caching`, `citations`, whether a `schema` was used, plus the per-operation ones (`dimensions`, `task_type`, `size`, `count`, `voice`, `format`, `language`, `pages`, `document_count`, `top_n`), `server_tools` and the provider's `server_tool_use` counters. `provider_options` is included, filtered twice: through the app's own parameter filter, and again against the credential-name matcher that catches `X-Api-Key` on a header -- an `api_key` passed per call sails straight through a password-shaped filter. For `operation: "tool"` this holds the tool result's class instead. |
+| `tool_call_id` | The provider's id for a tool invocation, for joining a tool call to the assistant turn that asked for it. |
+| `prompt` | Last user turn, only when `config.capture_llm_content` is on (off by default). Capped at 4 KiB of bytes. |
+| `completion` | The reply, same condition and cap. For `operation: "tool"` these two hold the tool's arguments and result instead. |
+
 ### `storage_op`
 
 Every Active Storage service operation. See

data/lib/railwatch/configuration.rb

@@ -5,7 +5,8 @@ module Railwatch
   # Laravel Nightwatch's config so the two products document the same knobs.
   class Configuration
     RECORD_TYPES = %i[queries cache_events mail broadcasts notifications outgoing_requests
-                      storage_ops view_renders logs transactions deprecations sessions].freeze
+                      storage_ops view_renders logs transactions deprecations sessions
+                      llm_calls].freeze
 
     # Framework/vendor noise excluded by default so a fresh install isn't
     # dominated by Rails' own housekeeping. Both lists are opt-in to disable
@@ -78,7 +79,8 @@ module Railwatch
                   :profile_sample, :profile_slow_ms, :profile_interval_us, :profiler,
                   :capture_job_arguments, :capture_job_retry_errors, :capture_response_body_on_error, :max_attachment_bytes,
                   :track_sessions, :session_flush_interval, :session_timeout,
-                  :capture_console, :interactive_runner_paths, :ignored_request_paths
+                  :capture_console, :interactive_runner_paths, :ignored_request_paths,
+                  :capture_llm_content
 
     attr_reader :deploy, :deploy_source, :detect_deploy, :user_resolver, :beacon_user_resolver,
                 :fingerprint_resolver, :redactors, :rejectors, :before_ingest, :backpressure_high_water
@@ -175,6 +177,10 @@ module Railwatch
       @capture_job_arguments = env_bool("RAILWATCH_CAPTURE_JOB_ARGUMENTS", false)
       @capture_job_retry_errors = env_bool("RAILWATCH_CAPTURE_JOB_RETRY_ERRORS", false)
       @capture_response_body_on_error = env_bool("RAILWATCH_CAPTURE_RESPONSE_BODY_ON_ERROR", false)
+      # Prompts and completions are whatever the app sent a provider, so
+      # they are off until an operator opts in. Token counts, model, and
+      # cost -- the reason the record exists -- are always captured.
+      @capture_llm_content = env_bool("RAILWATCH_CAPTURE_LLM_CONTENT", false)
       @max_attachment_bytes = env_int("RAILWATCH_MAX_ATTACHMENT_BYTES", 1_048_576)
       # Release health: one `session` record per browser tab (the beacon
       # client) and per authenticated/cookied server session (Railwatch::Sessions).

data/lib/railwatch/execution.rb

@@ -12,7 +12,7 @@ module Railwatch
     MAX_RECORDS = 10_000
     COUNTERS = %i[queries cached_queries exceptions logs cache_events jobs_enqueued mail
                   broadcasts notifications outgoing_requests storage_ops view_renders
-                  transactions hydrated_models lazy_loads deprecations spans].freeze
+                  transactions hydrated_models lazy_loads deprecations spans llm_calls].freeze
     # GC.stat with no key builds the whole stat hash; whether this Ruby
     # reports GC time never changes, so ask once.
     GC_TIME_SUPPORTED = GC.stat.key?(:time)

data/lib/railwatch/record.rb

@@ -10,7 +10,7 @@ module Railwatch
       query: 1, n_plus_one: 1, transaction: 1, exception: 1, cache_event: 1, mail: 1,
       broadcast: 1, notification: 1, outgoing_request: 1, storage_op: 1, view_render: 1,
       log: 1, enqueued_job: 1, user: 1, deprecation: 1, visit: 1, process: 1, span: 1, health: 1,
-      profile: 1, attachment: 1, session: 1
+      profile: 1, attachment: 1, session: 1, llm_call: 1
     }.freeze
 
     # Used in place of an execution's envelope when there is no execution, so

data/lib/railwatch/subscribers/llm.rb

@@ -0,0 +1,335 @@
+# frozen_string_literal: true
+
+module Railwatch
+  module Subscribers
+    # RubyLLM's own instrumentation. It emits an ActiveSupport::Notifications
+    # event per model call -- chat, embedding, image, and the rest -- plus one
+    # per tool invocation, so nothing here patches RubyLLM; we subscribe the
+    # same way we subscribe to Rails.
+    #
+    # One `llm_call` record per event, with `operation` naming which kind it
+    # was. Tool calls are the same record with operation "tool": they sit in
+    # the same execution waterfall and carry no tokens or cost.
+    #
+    # Two RubyLLM generations are supported. 1.16 puts token counts on the
+    # event as scalars and reports no cost at all; 2.0 sends Tokens and Cost
+    # objects built from its usage ledger. Both normalise to the same wire
+    # record, so a 1.16 app simply has no cost. Subscribing to an event
+    # RubyLLM never emits costs nothing, so the operations 2.0 added are
+    # subscribed unconditionally rather than behind a version check.
+    module Llm
+      extend Base
+
+      module_function
+
+      # Every usage-bearing operation in RubyLLM 2.0. The ones 1.16 knows
+      # about (chat, embedding, image, transcription, moderation) emit the
+      # same event names, so this list needs no version branch. `compaction`
+      # is a chat call by another name -- same payload, same tokens, same
+      # cost -- and is billed, so it belongs here rather than being invisible
+      # spend.
+      OPERATIONS = %w[chat compaction embedding image speech transcription
+                      moderation rerank ocr].freeze
+
+      # Costs are fractions of a cent: a cheap model's call is well under a
+      # microdollar, and floats summed across a month of rollups do not add
+      # up to an invoice. Nanodollars keep it exact in an integer column
+      # ($1,000 is 1e12, comfortably inside i64).
+      NANOS_PER_DOLLAR = 1_000_000_000
+
+      # Matches capture_response_body_on_error's cap. Prompts and completions
+      # are free text an app controls, so this is a size bound, not redaction.
+      CONTENT_MAX = 4096
+
+      def install!(_app)
+        OPERATIONS.each do |operation|
+          subscribe("#{operation}.ruby_llm") { |event| record_call(operation, event) }
+        end
+        subscribe("tool_call.ruby_llm") { |event| record_tool(event) }
+      end
+
+      def record_call(operation, event)
+        exe = execution
+        exe&.count(:llm_calls)
+        return unless recording?
+
+        p = event.payload
+        provider = p[:provider].to_s
+        model = p[:model].to_s
+        Railwatch.record(:llm_call,
+          group: Record.group_hash(provider, model, operation),
+          timestamp: started_at(event),
+          operation: operation,
+          provider: provider,
+          model: model,
+          response_model: p[:response_model]&.to_s&.slice(0, 255),
+          duration: micros(event),
+          streaming: p[:streaming] == true,
+          message_count: p[:message_count],
+          tool_count: Array(p[:tools]).size,
+          tools: tool_names(p),
+          cost_nanos: cost_nanos(p),
+          cost_reported: cost_reported(p),
+          finish_reason: finish_reason(p),
+          provider_request_id: provider_request_id(p),
+          params: params(operation, p),
+          **attachments(p),
+          **tokens(p),
+          **workflow(p),
+          **outcome(p),
+          prompt: content(prompt_text(p)),
+          completion: content(message_text(p[:response])))
+      end
+
+      # RubyLLM's opt-in tool_concurrency (:threads or :fibers) runs each
+      # tool in a fresh thread or fiber, and Current is backed by
+      # IsolatedExecutionState, which a new thread does not inherit. So this
+      # fires with no execution and the record is dropped.
+      #
+      # It cannot be fixed from here: by the time the event is delivered we
+      # are already inside the worker, with no reference to the execution
+      # that spawned it, and the thread is RubyLLM's to create
+      # (chat/tool_concurrency.rb propagates its own workflow context across
+      # that boundary, but knows nothing of ours). The same is true of every
+      # subscriber in an app-spawned thread. Dropping beats guessing: a
+      # process-wide fallback would file one request's tool call under
+      # another's execution. Concurrency is off by default, and the model
+      # calls are unaffected either way, so cost stays complete.
+      def record_tool(event)
+        exe = execution
+        exe&.count(:llm_calls)
+        return unless recording?
+
+        p = event.payload
+        tool_name = p[:tool_name].to_s
+        Railwatch.record(:llm_call,
+          group: Record.group_hash("tool", tool_name),
+          timestamp: started_at(event),
+          operation: "tool",
+          provider: p[:provider].to_s,
+          model: p[:model].to_s,
+          tool_name: tool_name[0, 255],
+          tool_call_id: p[:tool_call_id]&.to_s&.slice(0, 128),
+          params: ({ result_class: p[:result_class].to_s[0, 128] } if p[:result_class]),
+          duration: micros(event),
+          **workflow(p),
+          **outcome(p),
+          prompt: content(p[:tool_arguments]),
+          completion: content(p[:result_content]))
+      end
+
+      # Why the model stopped. :max_tokens means the answer was cut off --
+      # a truncated extraction reads exactly like a complete one without
+      # this, which is the failure most worth being able to see.
+      def finish_reason(payload)
+        response = payload[:response]
+        return nil unless response.respond_to?(:finish_reason)
+
+        response.finish_reason&.to_s&.slice(0, 32)
+      end
+
+      # Message#raw is the Faraday response (protocol.rb hands it in), so the
+      # provider's own request id is in its headers. It is what a provider
+      # support ticket asks for, and the only key that joins our record to
+      # theirs.
+      REQUEST_ID_HEADERS = %w[request-id x-request-id x-amzn-requestid].freeze
+
+      def provider_request_id(payload)
+        raw = payload[:response]
+        raw = raw.raw if raw.respond_to?(:raw)
+        headers = raw.respond_to?(:headers) ? raw.headers : nil
+        return nil unless headers.respond_to?(:[])
+
+        REQUEST_ID_HEADERS.each do |name|
+          value = headers[name]
+          return value.to_s[0, 128] if value.present?
+        end
+        nil
+      end
+
+      # Which tools the model could reach on this call. tool_count alone says
+      # how many; retracing needs which.
+      def tool_names(payload)
+        names = Array(payload[:tools]).map(&:to_s)
+        names.empty? ? nil : names.first(50).join(",")[0, 1024]
+      end
+
+      # Whether the provider priced the call itself, or we estimated it from
+      # the registry. The difference matters when a total is queried against
+      # an invoice.
+      def cost_reported(payload)
+        tokens = payload[:tokens]
+        return nil unless tokens.respond_to?(:reported_cost)
+        # No cost means no provenance to report. false would claim the
+        # registry priced it, which is the same false certainty cost_nanos
+        # avoids by being nil rather than zero.
+        return nil if cost_nanos(payload).nil?
+
+        !tokens.reported_cost.nil?
+      end
+
+      # What the call carried besides text. Images and PDFs are most of the
+      # input tokens on a document-reading call, and without this an
+      # expensive scan is indistinguishable from an expensive prompt.
+      # Only the last user turn is measured: earlier turns were counted by
+      # the calls that sent them, and walking the whole history would both
+      # double-count and cost O(messages) on every call.
+      def attachments(payload)
+        message = last_user_message(payload)
+        list = message.respond_to?(:attachments) ? Array(message.attachments) : []
+        return {} if list.empty?
+
+        types = list.filter_map { |a| a.type.to_s if a.respond_to?(:type) }.tally
+          .sort_by { |_, n| -n }.map { |type, n| n > 1 ? "#{type}x#{n}" : type }.join(",")
+        { attachments: list.size, attachment_types: types[0, 128],
+          attachment_names: content(list.filter_map { |a| a.filename if a.respond_to?(:filename) }.join(", ")) }
+      end
+
+      # The knobs that change what a call costs and what it returns, so a
+      # surprising result can be reproduced with the settings that produced
+      # it. Provider options go through the app's own parameter filter: they
+      # are request configuration, but an app can put anything in them.
+      COMMON_PARAMS = %i[temperature max_output_tokens tool_choice tool_call_limit
+                         thinking caching citations dimensions task_type size count
+                         voice format language pages document_count top_n].freeze
+
+      def params(operation, payload)
+        out = {}
+        COMMON_PARAMS.each do |key|
+          value = payload[key]
+          next if value.nil?
+          # false is kept, not dropped: `caching` defaults to nil, so
+          # caching: false is a deliberate choice, and reproducing a call
+          # needs the settings it ran with. RubyLLM does not distinguish a
+          # boolean that was set from one that defaulted, so record both
+          # rather than guess which mattered.
+          out[key] = value.is_a?(Numeric) || [ true, false ].include?(value) ? value : value.to_s[0, 128]
+        end
+        out[:schema] = true if payload[:schema]
+        out[:server_tools] = Array(payload[:server_tools]).map(&:to_s).first(20) if payload[:server_tools].present?
+        if (usage = payload[:tokens]).respond_to?(:server_tool_use) && usage.server_tool_use.present?
+          out[:server_tool_use] = usage.server_tool_use
+        end
+        if (options = payload[:provider_options]).is_a?(Hash) && !options.empty?
+          out[:provider_options] = provider_options(options)
+        end
+        out[:operation] = operation unless out.empty?
+        out.empty? ? nil : out
+      end
+
+      # Two filters, because one is not enough here. The app's parameter
+      # filter defaults to password-shaped names only, and provider_options
+      # is the one place in this payload where a per-request credential
+      # plausibly lives -- an api_key passed per call sails straight through
+      # a password filter. The redactor's credential-name matcher (the same
+      # one that catches X-Api-Key on a header) closes that.
+      def provider_options(options)
+        redact_credentials(Railwatch.redactor.params(options.transform_keys(&:to_s)))
+      end
+
+      # Recursive, because provider options nest: extra_headers carrying an
+      # authorization value is a hash inside the hash, and a top-level scan
+      # walks straight past it.
+      def redact_credentials(value, depth = 0)
+        return value if depth > 4
+
+        case value
+        when Hash
+          value.each_with_object({}) do |(key, item), out|
+            # The matcher is written for header names, which are hyphenated;
+            # provider options are Ruby-ish and use underscores, so api_key
+            # would sail past a pattern expecting api-key.
+            out[key] = if Railwatch.redactor.redact_header?(key.to_s.tr("_", "-"))
+              Redactor::FILTERED
+            else
+              redact_credentials(item, depth + 1)
+            end
+          end
+        when Array then value.map { |item| redact_credentials(item, depth + 1) }
+        else value
+        end
+      end
+
+      def last_user_message(payload)
+        messages = payload[:input_messages]
+        return nil unless messages.respond_to?(:reverse_each)
+
+        messages.reverse_each.find { |m| m.respond_to?(:role) && m.role.to_s == "user" }
+      end
+
+      # 2.0 sends a RubyLLM::Tokens; 1.16 sends bare counts on the event.
+      def tokens(payload)
+        counts = payload[:tokens]
+        if counts.respond_to?(:input)
+          { input_tokens: counts.input, output_tokens: counts.output,
+            cache_read_tokens: counts.cache_read, cache_write_tokens: counts.cache_write,
+            thinking_tokens: counts.thinking }
+        else
+          { input_tokens: payload[:input_tokens], output_tokens: payload[:output_tokens],
+            cache_read_tokens: payload[:cached_tokens], cache_write_tokens: payload[:cache_creation_tokens],
+            thinking_tokens: payload[:thinking_tokens] }
+        end
+      end
+
+      # nil in three distinct cases that all mean "we do not know": RubyLLM
+      # 1.16 (which reports no cost), a model the registry has no pricing
+      # for, and an operation that used no tokens. Never zero -- an unpriced
+      # call is not a free call, and the difference matters on a bill.
+      def cost_nanos(payload)
+        cost = payload[:cost]
+        return nil unless cost.respond_to?(:total)
+
+        total = cost.total
+        total && (total * NANOS_PER_DOLLAR).round
+      end
+
+      # Present only on 2.0, and only inside RubyLLM.workflow. Stamped on
+      # every event the block emits, which is what lets an agent run be
+      # reassembled from its steps.
+      def workflow(payload)
+        return {} unless payload[:workflow_id]
+
+        { workflow_id: payload[:workflow_id].to_s[0, 64],
+          workflow_name: payload[:workflow_name].to_s[0, 255],
+          workflow_step_id: payload[:workflow_step_id]&.to_s&.slice(0, 64),
+          workflow_step_name: payload[:workflow_step_name]&.to_s&.slice(0, 255),
+          workflow_step_parent_id: payload[:workflow_step_parent_id]&.to_s&.slice(0, 64) }
+      end
+
+      # Rails adds :exception to the payload when the instrumented block
+      # raised; RubyLLM leaves the rest of the event untouched in that case.
+      def outcome(payload)
+        error = payload[:exception]
+        return { status: "ok" } unless error
+
+        { status: "failed", error: "#{Array(error).first}: #{Array(error).last}"[0, 255] }
+      end
+
+      # The last thing the app asked, which is the half of a conversation
+      # worth seeing next to a cost. Earlier turns are the app's own records.
+      def prompt_text(payload)
+        return payload[:input] || payload[:prompt] || payload[:query] unless payload[:input_messages].respond_to?(:reverse_each)
+
+        message_text(last_user_message(payload))
+      end
+
+      def message_text(message)
+        return nil if message.nil?
+
+        message.respond_to?(:content) ? message.content : message
+      end
+
+      def content(value)
+        return nil unless Railwatch.config.capture_llm_content
+        return nil if value.nil?
+
+        text = value.to_s
+        return nil if text.empty?
+        # byteslice, not [0, n]: the cap bounds what is buffered and shipped,
+        # and 4096 characters of CJK is three times that in bytes. scrub
+        # repairs the multibyte character the slice may have cut in half.
+        text.bytesize > CONTENT_MAX ? text.byteslice(0, CONTENT_MAX).scrub("") : text
+      end
+    end
+  end
+end

data/lib/railwatch/subscribers.rb

@@ -12,6 +12,7 @@ require "railwatch/subscribers/storage"
 require "railwatch/subscribers/views"
 require "railwatch/subscribers/logs"
 require "railwatch/subscribers/jobs"
+require "railwatch/subscribers/llm"
 require "railwatch/subscribers/deprecations"
 require "railwatch/subscribers/users"
 require "railwatch/subscribers/process_info"
@@ -19,7 +20,7 @@ require "railwatch/subscribers/process_info"
 module Railwatch
   module Subscribers
     ALL = [ Requests, Queries, Exceptions, Cache, Mail, Broadcasts, Notifications,
-            Storage, Views, Logs, Jobs, Deprecations, Users, ProcessInfo ].freeze
+            Storage, Views, Logs, Jobs, Deprecations, Users, ProcessInfo, Llm ].freeze
 
     module_function
 

data/lib/railwatch/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Railwatch
-  VERSION = "0.1.2"
+  VERSION = "0.1.4"
 end

data/lib/railwatch.rb

@@ -496,7 +496,7 @@ module Railwatch
       query: :queries, n_plus_one: :queries, transaction: :transactions, cache_event: :cache_events,
       mail: :mail, broadcast: :broadcasts, notification: :notifications, outgoing_request: :outgoing_requests,
       storage_op: :storage_ops, view_render: :view_renders, log: :logs, deprecation: :deprecations,
-      session: :sessions
+      session: :sessions, llm_call: :llm_calls
     }.freeze
 
     def type_plural(type)

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: railwatch
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.1.4
 platform: ruby
 authors:
 - Cole Robertson
 bindir: bin
 cert_chain: []
-date: 2026-09-15 00:00:00.000000000 Z
+date: 2026-09-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -115,6 +115,7 @@ files:
 - lib/railwatch/subscribers/deprecations.rb
 - lib/railwatch/subscribers/exceptions.rb
 - lib/railwatch/subscribers/jobs.rb
+- lib/railwatch/subscribers/llm.rb
 - lib/railwatch/subscribers/logs.rb
 - lib/railwatch/subscribers/mail.rb
 - lib/railwatch/subscribers/notifications.rb