railwatch 0.1.1 → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 27a48b972fe7616fbf46c861f8b0f1a9ffb4d99146ec2121cb05274ea3dee031
4
- data.tar.gz: 45c879afc12f731e446be6aa43f8a78b61d078a8f5a7cdc6294bd94c4dfe864e
3
+ metadata.gz: 41585458ddfaf2036a184f5afca05be24749ed73ba54bb41f38ca7e881694ac9
4
+ data.tar.gz: f4bd36a26ec1eb80ee729596c116015f3159f6ecb4306d64ebce9ae3f1eb4335
5
5
  SHA512:
6
- metadata.gz: 29512ddbb519a85a1334531289be33d59d4ceb3cf0a8c7ef55d3db04ab41ae884adf65b64ddecd40d1b70254ca76ed0ce0bbb38ec03eee0125dbae5a8b7730de
7
- data.tar.gz: 67d7a5573b95aa032a227a8dc10e440bfaee6681628de90d2948f074ceb15ffb9ea839968984beaaec545d0db161a266ba94e99cd2b3a0908c14c9e98969c2c7
6
+ metadata.gz: 67197b5696dd6d041e8f8a311ac7b5ca6b9f989347dd1c45b542e5cc3a5ad70207af7c507d135c721a522755dde589b8bbe16c912e163bf7796ed2bb6a6fcbed
7
+ data.tar.gz: e3e3746abd37ebfec16b09dae18d67f8ba1ef425b1a4571c805102f313eb93dca19558d64ba3212d18fb7eba4e7cbdc18c670f264cc9288ca65d3fea81b919c7
data/CHANGELOG.md CHANGED
@@ -1,5 +1,46 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.1.3 (2026-09-15)
4
+
5
+ - LLM calls are recorded from RubyLLM's own instrumentation. Every model
6
+ call it emits -- `chat`, `compaction`, `embedding`, `image`, `speech`,
7
+ `transcription`, `moderation`, `rerank`, `ocr` -- plus each tool
8
+ invocation becomes one
9
+ `llm_call` child record on the request, job, or command that made it,
10
+ carrying provider, model, duration, token counts per bucket, and cost.
11
+ Nothing is patched: RubyLLM publishes ActiveSupport::Notifications events
12
+ and Railwatch subscribes to them like any Rails event.
13
+ - Both RubyLLM generations are read from the same subscriber. 1.16 puts
14
+ token counts on the event as scalars and reports no cost; 2.0 sends its
15
+ `Tokens` and `Cost` objects, and stamps `workflow_id` and step identity on
16
+ every event inside `RubyLLM.workflow`, which is recorded so an agent run
17
+ can be reassembled from its steps. A 1.16 app has no cost rather than a
18
+ cost of zero, and a model the registry cannot price is unpriced, not free.
19
+ - `capture_llm_content` (default off, `RAILWATCH_CAPTURE_LLM_CONTENT`)
20
+ records the last user turn and the reply, capped at 4 KiB of bytes each. Token
21
+ counts, model, and cost are always captured; prompts are not, because
22
+ they are whatever the app sent a provider.
23
+
24
+ ## 0.1.2 (2026-09-14)
25
+
26
+ - A failed job's exception is reported once. Solid Queue re-raises it out
27
+ of the worker thread, where its app executor reports the same error object
28
+ to `Rails.error` again after the `job_attempt` execution has finished; that
29
+ second report (source `application.solid_queue`, unlinked) doubled every
30
+ failed job's occurrence count. An error object now remembers that its
31
+ unhandled report has shipped, across executions.
32
+ - `require "railwatch/minitest"` includes the assertions into
33
+ `ActiveSupport::TestCase` through its load hook, so `assert_railwatch_queries`
34
+ works after the generator's one-line edit to `test/test_helper.rb` without
35
+ a manual `include`.
36
+ - Solid Queue's supervisor, dispatcher, scheduler, and forked workers report
37
+ role `worker` after Solid Queue renames the process. They were classified
38
+ `web` by every health sample taken after boot, with empty Puma thread
39
+ stats.
40
+ - Troubleshooting entries for `json` 3.0 against Rails 8.1.3.1 (`bin/jobs`
41
+ crash loop, not a Railwatch fault) and for deprecations that are counted
42
+ but never listed because the app's deprecation behavior lacks `:notify`.
43
+
3
44
  ## 0.1.1 (2026-09-14)
4
45
 
5
46
  - Token prefixes are `rw_` for an environment's ingest token and `rwp_`
data/docs/records.md CHANGED
@@ -502,6 +502,63 @@ never double-recorded.
502
502
  | `response_body` | First 4 KiB of the response body, but only when `config.capture_response_body_on_error` is on (off by default) *and* the response was an error. A body that parses as a JSON object is run through the same parameter filter as request params and re-serialized; anything else is stored as it arrived. nil in every other case — including a connection failure, where there is no response (on the Net::HTTP path a body is read only if Net::HTTP already buffered it, so a response being streamed through `read_body` is never consumed; on the Faraday path the body is taken only once a status came back, so an outgoing request payload can never be filed as a response). |
503
503
  | `source` | App-code call site (Net::HTTP path only). |
504
504
 
505
+ ### `llm_call`
506
+
507
+ Every RubyLLM model call and tool invocation, from
508
+ `lib/railwatch/subscribers/llm.rb`. RubyLLM publishes its own
509
+ `ActiveSupport::Notifications` events, so nothing is patched and RubyLLM is
510
+ not a dependency — an app without it never emits these. Requires RubyLLM
511
+ 1.16 or later, which is where its instrumentation landed.
512
+
513
+ The model call also appears as an `outgoing_request`, since it is an HTTP
514
+ call like any other. The two are different grains on purpose: the
515
+ `outgoing_request` is the HTTP truth, the `llm_call` is what it cost.
516
+
517
+ **Token counts and cost differ by RubyLLM version.** 1.16 reports token
518
+ counts and no cost at all. 2.0 reports both, from its usage ledger, and
519
+ adds the `workflow_*` fields. `cost_nanos` is null rather than zero
520
+ whenever RubyLLM reported no cost or the model registry could not price
521
+ it — an unpriced call is not a free one.
522
+
523
+ **Concurrent tool calls are not recorded.** RubyLLM's opt-in
524
+ `tool_concurrency` (`:threads` or `:fibers`) runs each tool in a fresh
525
+ thread or fiber. `Railwatch::Current` is backed by
526
+ `ActiveSupport::IsolatedExecutionState`, which a new thread does not
527
+ inherit, so the `tool_call.ruby_llm` event fires with no execution to
528
+ attach to and the record is dropped rather than misattributed. This
529
+ affects every Railwatch subscriber in an app-spawned thread, not just this
530
+ one. Tool concurrency is off by default; with it off, tool calls are
531
+ recorded normally. The model calls themselves are unaffected either way,
532
+ so cost is always complete.
533
+
534
+ | Field | Meaning |
535
+ |---|---|
536
+ | `group` | Hash of provider + model + operation, or of `"tool"` + tool name. |
537
+ | `operation` | `"chat"`, `"compaction"`, `"embedding"`, `"image"`, `"speech"`, `"transcription"`, `"moderation"`, `"rerank"`, `"ocr"`, or `"tool"`. |
538
+ | `provider` | Provider slug, e.g. `"anthropic"`. |
539
+ | `model` | Model the call was made with. Empty for a provider that selects its own (moderation). |
540
+ | `response_model` | Model the provider says answered, which can differ from the one asked for. |
541
+ | `tool_name` | Tool name, for `operation: "tool"`. |
542
+ | `duration` | Microseconds. |
543
+ | `status` | `"ok"`, or `"failed"` if the call raised. |
544
+ | `error` | `"Class: message"`, truncated to 255 chars, if the call raised. |
545
+ | `streaming` | Whether the call was streamed. |
546
+ | `message_count` | Conversation length at the time of the call. |
547
+ | `tool_count` | Number of tools the model was offered. |
548
+ | `input_tokens` | Standard (non-cached) input tokens. |
549
+ | `output_tokens` | Billable output tokens. |
550
+ | `cache_read_tokens` | Tokens served from the provider's prompt cache. |
551
+ | `cache_write_tokens` | Tokens written to the provider's prompt cache. |
552
+ | `thinking_tokens` | Reasoning tokens, where the provider reports them separately. |
553
+ | `cost_nanos` | Cost in billionths of a US dollar. Null when unpriced — see above. Nanodollars because a cheap call is well under a microdollar and floats do not sum to an invoice. |
554
+ | `workflow_id` | `RubyLLM.workflow` identifier (2.0+). Null outside a workflow. |
555
+ | `workflow_name` | Workflow name (2.0+). |
556
+ | `workflow_step_id` | Step identifier within the workflow (2.0+). |
557
+ | `workflow_step_name` | Step name (2.0+). |
558
+ | `workflow_step_parent_id` | Enclosing step, for nested steps — what reconstructs the tree (2.0+). |
559
+ | `prompt` | Last user turn, only when `config.capture_llm_content` is on (off by default). Capped at 4 KiB of bytes. |
560
+ | `completion` | The reply, same condition and cap. For `operation: "tool"` these two hold the tool's arguments and result instead. |
561
+
505
562
  ### `storage_op`
506
563
 
507
564
  Every Active Storage service operation. See
@@ -652,7 +709,9 @@ ship. Forked workers start with an empty cache.
652
709
 
653
710
  ### `deprecation`
654
711
 
655
- `deprecation.rails`. See `lib/railwatch/subscribers/deprecations.rb`.
712
+ `deprecation.rails`. See `lib/railwatch/subscribers/deprecations.rb`. Rails
713
+ only emits that notification when `config.active_support.deprecation`
714
+ includes `:notify`; see [Troubleshooting](troubleshooting.md#deprecations-are-counted-but-never-listed).
656
715
 
657
716
  | Field | Meaning |
658
717
  |---|---|
@@ -12,12 +12,12 @@ the other.
12
12
 
13
13
  ## Decide whether Railwatch covers your workload
14
14
 
15
- Nightwatch parity and Sentry parity are different targets. Laravel
16
- Nightwatch is an application-monitoring product built around framework
17
- executions; Railwatch deliberately maps that model onto Rails. Sentry is a
18
- broader, multi-language managed platform with browser replay, native/mobile
19
- SDKs, a large integration catalog, and generic tracing. Those broader
20
- capabilities are not implied by Rails/Nightwatch parity.
15
+ Railwatch and Sentry aim at different targets. Railwatch is an
16
+ application-monitoring product built around Rails executions: a request,
17
+ job, scheduled task, command, or channel action and everything it did.
18
+ Sentry is a broader, multi-language managed platform with browser replay,
19
+ native/mobile SDKs, a large integration catalog, and generic tracing.
20
+ Railwatch does not claim those broader capabilities.
21
21
 
22
22
  | Workload or capability | Status | Boundary |
23
23
  |---|---|---|
data/docs/testing.md CHANGED
@@ -18,15 +18,12 @@ require "railwatch/rspec"
18
18
  That requires `railwatch/spec_helper`, includes `Railwatch::SpecHelper` into every
19
19
  example group, and defines the matchers below.
20
20
 
21
- Minitest: the same thing in `test/test_helper.rb`.
21
+ Minitest: the same thing in `test/test_helper.rb`. The require includes
22
+ `Railwatch::Minitest` into `ActiveSupport::TestCase` on its own.
22
23
 
23
24
  ```ruby
24
25
  require "rails/test_help"
25
26
  require "railwatch/minitest"
26
-
27
- class ActiveSupport::TestCase
28
- include Railwatch::Minitest
29
- end
30
27
  ```
31
28
 
32
29
  Railwatch must be *enabled* in the test environment or every block would look
@@ -81,6 +81,42 @@ to stderr prefixed `[railwatch]`. They never go to `Rails.logger`, so they
81
81
  can't become `log` records about themselves. `Railwatch.on_unrecoverable
82
82
  { |e| ... }` gets the same failures as a callback.
83
83
 
84
+ ## `bin/jobs` dies in a loop with `wrong number of arguments (given 2, expected 1)`
85
+
86
+ **Symptom.** Every Solid Queue process crashes at boot from
87
+ `json/common.rb` in `parse`, and Railwatch reports the same
88
+ `ArgumentError` from source `application.solid_queue` hundreds of times.
89
+ The web server still answers until it touches a session or a JSON column.
90
+
91
+ **Cause.** Not Railwatch. `json` 3.0 (September 2026) made `JSON.parse`'s
92
+ options keyword-only, and Active Support up to 8.1.3.1 still passes them
93
+ positionally ([rails/rails#58685](https://github.com/rails/rails/issues/58685)).
94
+ A fresh `rails new` resolves the newest `json`, so a new app hits this
95
+ before Railwatch is even installed.
96
+
97
+ **Fix.** Pin `json` below 3 until a Rails release includes the fix:
98
+
99
+ ```sh
100
+ bundle add json --version "< 3"
101
+ ```
102
+
103
+ ## Deprecations are counted but never listed
104
+
105
+ **Symptom.** The `deprecations` counter on an execution is non-zero, but
106
+ the Deprecations page stays empty.
107
+
108
+ **Cause.** Railwatch listens to the `deprecation.rails` notification, and
109
+ Rails only emits it when the app's deprecation behavior includes
110
+ `:notify`. The default in development is `:log` and in production
111
+ `:silence`, neither of which notifies.
112
+
113
+ **Fix.** Add `:notify` alongside whatever the environment already does:
114
+
115
+ ```ruby
116
+ # config/environments/production.rb
117
+ config.active_support.deprecation = [:silence, :notify]
118
+ ```
119
+
84
120
  ## Doubled scheduled_task records
85
121
 
86
122
  **Symptom.** Every recurring task shows twice on the Scheduled tasks
@@ -5,7 +5,8 @@ module Railwatch
5
5
  # Laravel Nightwatch's config so the two products document the same knobs.
6
6
  class Configuration
7
7
  RECORD_TYPES = %i[queries cache_events mail broadcasts notifications outgoing_requests
8
- storage_ops view_renders logs transactions deprecations sessions].freeze
8
+ storage_ops view_renders logs transactions deprecations sessions
9
+ llm_calls].freeze
9
10
 
10
11
  # Framework/vendor noise excluded by default so a fresh install isn't
11
12
  # dominated by Rails' own housekeeping. Both lists are opt-in to disable
@@ -78,7 +79,8 @@ module Railwatch
78
79
  :profile_sample, :profile_slow_ms, :profile_interval_us, :profiler,
79
80
  :capture_job_arguments, :capture_job_retry_errors, :capture_response_body_on_error, :max_attachment_bytes,
80
81
  :track_sessions, :session_flush_interval, :session_timeout,
81
- :capture_console, :interactive_runner_paths, :ignored_request_paths
82
+ :capture_console, :interactive_runner_paths, :ignored_request_paths,
83
+ :capture_llm_content
82
84
 
83
85
  attr_reader :deploy, :deploy_source, :detect_deploy, :user_resolver, :beacon_user_resolver,
84
86
  :fingerprint_resolver, :redactors, :rejectors, :before_ingest, :backpressure_high_water
@@ -175,6 +177,10 @@ module Railwatch
175
177
  @capture_job_arguments = env_bool("RAILWATCH_CAPTURE_JOB_ARGUMENTS", false)
176
178
  @capture_job_retry_errors = env_bool("RAILWATCH_CAPTURE_JOB_RETRY_ERRORS", false)
177
179
  @capture_response_body_on_error = env_bool("RAILWATCH_CAPTURE_RESPONSE_BODY_ON_ERROR", false)
180
+ # Prompts and completions are whatever the app sent a provider, so
181
+ # they are off until an operator opts in. Token counts, model, and
182
+ # cost -- the reason the record exists -- are always captured.
183
+ @capture_llm_content = env_bool("RAILWATCH_CAPTURE_LLM_CONTENT", false)
178
184
  @max_attachment_bytes = env_int("RAILWATCH_MAX_ATTACHMENT_BYTES", 1_048_576)
179
185
  # Release health: one `session` record per browser tab (the beacon
180
186
  # client) and per authenticated/cookied server session (Railwatch::Sessions).
@@ -12,7 +12,7 @@ module Railwatch
12
12
  MAX_RECORDS = 10_000
13
13
  COUNTERS = %i[queries cached_queries exceptions logs cache_events jobs_enqueued mail
14
14
  broadcasts notifications outgoing_requests storage_ops view_renders
15
- transactions hydrated_models lazy_loads deprecations spans].freeze
15
+ transactions hydrated_models lazy_loads deprecations spans llm_calls].freeze
16
16
  # GC.stat with no key builds the whole stat hash; whether this Ruby
17
17
  # reports GC time never changes, so ask once.
18
18
  GC_TIME_SUPPORTED = GC.stat.key?(:time)
@@ -7,10 +7,9 @@ module Railwatch
7
7
  # assert_/refute_. Add to test/test_helper.rb:
8
8
  #
9
9
  # require "railwatch/minitest"
10
- # class ActiveSupport::TestCase
11
- # include Railwatch::Minitest
12
- # end
13
10
  #
11
+ # That includes the module into ActiveSupport::TestCase (through its load
12
+ # hook, so the require order against rails/test_help does not matter).
14
13
  # Includes Railwatch::SpecHelper, so `railwatch_records(:query)` is available
15
14
  # too. See docs/testing.md.
16
15
  module Minitest
@@ -41,3 +40,5 @@ module Railwatch
41
40
  end
42
41
  end
43
42
  end
43
+
44
+ ActiveSupport.on_load(:active_support_test_case) { include Railwatch::Minitest } if defined?(ActiveSupport.on_load)
@@ -10,7 +10,7 @@ module Railwatch
10
10
  query: 1, n_plus_one: 1, transaction: 1, exception: 1, cache_event: 1, mail: 1,
11
11
  broadcast: 1, notification: 1, outgoing_request: 1, storage_op: 1, view_render: 1,
12
12
  log: 1, enqueued_job: 1, user: 1, deprecation: 1, visit: 1, process: 1, span: 1, health: 1,
13
- profile: 1, attachment: 1, session: 1
13
+ profile: 1, attachment: 1, session: 1, llm_call: 1
14
14
  }.freeze
15
15
 
16
16
  # Used in place of an execution's envelope when there is no execution, so
@@ -87,6 +87,13 @@ module Railwatch
87
87
  def capture(error, handled:, severity:, context: {}, source: nil, fingerprint: nil)
88
88
  return unless Railwatch.enabled?
89
89
  return if ignored?(error)
90
+ # Solid Queue re-raises a failed job's error out of the worker thread,
91
+ # where its executor reports it to Rails.error a second time
92
+ # (source application.solid_queue) after the job_attempt execution
93
+ # has already finished and reported it. The execution's own
94
+ # first_exception_report? bookkeeping is gone by then, so the error
95
+ # object itself remembers that its unhandled report has shipped.
96
+ return if !handled && reported_unhandled?(error)
90
97
 
91
98
  exe = execution
92
99
  if exe&.first_exception_observation?(error, handled)
@@ -151,9 +158,22 @@ module Railwatch
151
158
  # children (Railwatch.tail_keep?).
152
159
  exe.exception_reported = true if exe
153
160
  Railwatch.record_now(:exception, group: group, **rec)
161
+ remember_reported(error)
154
162
  end
155
163
  end
156
164
 
165
+ def reported_unhandled?(error)
166
+ error.instance_variable_defined?(:@__railwatch_reported)
167
+ rescue StandardError
168
+ false
169
+ end
170
+
171
+ def remember_reported(error)
172
+ error.instance_variable_set(:@__railwatch_reported, true)
173
+ rescue StandardError
174
+ nil
175
+ end
176
+
157
177
  # The group hash `capture` would assign this error. Public so
158
178
  # Railwatch.attach can file an attachment against the same issue without
159
179
  # having to re-derive the bucketing rule (source snippets are skipped:
@@ -0,0 +1,190 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Railwatch
4
+ module Subscribers
5
+ # RubyLLM's own instrumentation. It emits an ActiveSupport::Notifications
6
+ # event per model call -- chat, embedding, image, and the rest -- plus one
7
+ # per tool invocation, so nothing here patches RubyLLM; we subscribe the
8
+ # same way we subscribe to Rails.
9
+ #
10
+ # One `llm_call` record per event, with `operation` naming which kind it
11
+ # was. Tool calls are the same record with operation "tool": they sit in
12
+ # the same execution waterfall and carry no tokens or cost.
13
+ #
14
+ # Two RubyLLM generations are supported. 1.16 puts token counts on the
15
+ # event as scalars and reports no cost at all; 2.0 sends Tokens and Cost
16
+ # objects built from its usage ledger. Both normalise to the same wire
17
+ # record, so a 1.16 app simply has no cost. Subscribing to an event
18
+ # RubyLLM never emits costs nothing, so the operations 2.0 added are
19
+ # subscribed unconditionally rather than behind a version check.
20
+ module Llm
21
+ extend Base
22
+
23
+ module_function
24
+
25
+ # Every usage-bearing operation in RubyLLM 2.0. The ones 1.16 knows
26
+ # about (chat, embedding, image, transcription, moderation) emit the
27
+ # same event names, so this list needs no version branch. `compaction`
28
+ # is a chat call by another name -- same payload, same tokens, same
29
+ # cost -- and is billed, so it belongs here rather than being invisible
30
+ # spend.
31
+ OPERATIONS = %w[chat compaction embedding image speech transcription
32
+ moderation rerank ocr].freeze
33
+
34
+ # Costs are fractions of a cent: a cheap model's call is well under a
35
+ # microdollar, and floats summed across a month of rollups do not add
36
+ # up to an invoice. Nanodollars keep it exact in an integer column
37
+ # ($1,000 is 1e12, comfortably inside i64).
38
+ NANOS_PER_DOLLAR = 1_000_000_000
39
+
40
+ # Matches capture_response_body_on_error's cap. Prompts and completions
41
+ # are free text an app controls, so this is a size bound, not redaction.
42
+ CONTENT_MAX = 4096
43
+
44
+ def install!(_app)
45
+ OPERATIONS.each do |operation|
46
+ subscribe("#{operation}.ruby_llm") { |event| record_call(operation, event) }
47
+ end
48
+ subscribe("tool_call.ruby_llm") { |event| record_tool(event) }
49
+ end
50
+
51
+ def record_call(operation, event)
52
+ exe = execution
53
+ exe&.count(:llm_calls)
54
+ return unless recording?
55
+
56
+ p = event.payload
57
+ provider = p[:provider].to_s
58
+ model = p[:model].to_s
59
+ Railwatch.record(:llm_call,
60
+ group: Record.group_hash(provider, model, operation),
61
+ timestamp: started_at(event),
62
+ operation: operation,
63
+ provider: provider,
64
+ model: model,
65
+ response_model: p[:response_model]&.to_s&.slice(0, 255),
66
+ duration: micros(event),
67
+ streaming: p[:streaming] == true,
68
+ message_count: p[:message_count],
69
+ tool_count: Array(p[:tools]).size,
70
+ cost_nanos: cost_nanos(p),
71
+ **tokens(p),
72
+ **workflow(p),
73
+ **outcome(p),
74
+ prompt: content(prompt_text(p)),
75
+ completion: content(message_text(p[:response])))
76
+ end
77
+
78
+ # RubyLLM's opt-in tool_concurrency (:threads or :fibers) runs each
79
+ # tool in a fresh thread or fiber, and Current is backed by
80
+ # IsolatedExecutionState, which a new thread does not inherit. So this
81
+ # fires with no execution and the record is dropped.
82
+ #
83
+ # It cannot be fixed from here: by the time the event is delivered we
84
+ # are already inside the worker, with no reference to the execution
85
+ # that spawned it, and the thread is RubyLLM's to create
86
+ # (chat/tool_concurrency.rb propagates its own workflow context across
87
+ # that boundary, but knows nothing of ours). The same is true of every
88
+ # subscriber in an app-spawned thread. Dropping beats guessing: a
89
+ # process-wide fallback would file one request's tool call under
90
+ # another's execution. Concurrency is off by default, and the model
91
+ # calls are unaffected either way, so cost stays complete.
92
+ def record_tool(event)
93
+ exe = execution
94
+ exe&.count(:llm_calls)
95
+ return unless recording?
96
+
97
+ p = event.payload
98
+ tool_name = p[:tool_name].to_s
99
+ Railwatch.record(:llm_call,
100
+ group: Record.group_hash("tool", tool_name),
101
+ timestamp: started_at(event),
102
+ operation: "tool",
103
+ provider: p[:provider].to_s,
104
+ model: p[:model].to_s,
105
+ tool_name: tool_name[0, 255],
106
+ duration: micros(event),
107
+ **workflow(p),
108
+ **outcome(p),
109
+ prompt: content(p[:tool_arguments]),
110
+ completion: content(p[:result_content]))
111
+ end
112
+
113
+ # 2.0 sends a RubyLLM::Tokens; 1.16 sends bare counts on the event.
114
+ def tokens(payload)
115
+ counts = payload[:tokens]
116
+ if counts.respond_to?(:input)
117
+ { input_tokens: counts.input, output_tokens: counts.output,
118
+ cache_read_tokens: counts.cache_read, cache_write_tokens: counts.cache_write,
119
+ thinking_tokens: counts.thinking }
120
+ else
121
+ { input_tokens: payload[:input_tokens], output_tokens: payload[:output_tokens],
122
+ cache_read_tokens: payload[:cached_tokens], cache_write_tokens: payload[:cache_creation_tokens],
123
+ thinking_tokens: payload[:thinking_tokens] }
124
+ end
125
+ end
126
+
127
+ # nil in three distinct cases that all mean "we do not know": RubyLLM
128
+ # 1.16 (which reports no cost), a model the registry has no pricing
129
+ # for, and an operation that used no tokens. Never zero -- an unpriced
130
+ # call is not a free call, and the difference matters on a bill.
131
+ def cost_nanos(payload)
132
+ cost = payload[:cost]
133
+ return nil unless cost.respond_to?(:total)
134
+
135
+ total = cost.total
136
+ total && (total * NANOS_PER_DOLLAR).round
137
+ end
138
+
139
+ # Present only on 2.0, and only inside RubyLLM.workflow. Stamped on
140
+ # every event the block emits, which is what lets an agent run be
141
+ # reassembled from its steps.
142
+ def workflow(payload)
143
+ return {} unless payload[:workflow_id]
144
+
145
+ { workflow_id: payload[:workflow_id].to_s[0, 64],
146
+ workflow_name: payload[:workflow_name].to_s[0, 255],
147
+ workflow_step_id: payload[:workflow_step_id]&.to_s&.slice(0, 64),
148
+ workflow_step_name: payload[:workflow_step_name]&.to_s&.slice(0, 255),
149
+ workflow_step_parent_id: payload[:workflow_step_parent_id]&.to_s&.slice(0, 64) }
150
+ end
151
+
152
+ # Rails adds :exception to the payload when the instrumented block
153
+ # raised; RubyLLM leaves the rest of the event untouched in that case.
154
+ def outcome(payload)
155
+ error = payload[:exception]
156
+ return { status: "ok" } unless error
157
+
158
+ { status: "failed", error: "#{Array(error).first}: #{Array(error).last}"[0, 255] }
159
+ end
160
+
161
+ # The last thing the app asked, which is the half of a conversation
162
+ # worth seeing next to a cost. Earlier turns are the app's own records.
163
+ def prompt_text(payload)
164
+ messages = payload[:input_messages]
165
+ return payload[:input] || payload[:prompt] || payload[:query] unless messages.respond_to?(:reverse_each)
166
+
167
+ last = messages.reverse_each.find { |m| m.respond_to?(:role) && m.role.to_s == "user" }
168
+ message_text(last)
169
+ end
170
+
171
+ def message_text(message)
172
+ return nil if message.nil?
173
+
174
+ message.respond_to?(:content) ? message.content : message
175
+ end
176
+
177
+ def content(value)
178
+ return nil unless Railwatch.config.capture_llm_content
179
+ return nil if value.nil?
180
+
181
+ text = value.to_s
182
+ return nil if text.empty?
183
+ # byteslice, not [0, n]: the cap bounds what is buffered and shipped,
184
+ # and 4096 characters of CJK is three times that in bytes. scrub
185
+ # repairs the multibyte character the slice may have cut in half.
186
+ text.bytesize > CONTENT_MAX ? text.byteslice(0, CONTENT_MAX).scrub("") : text
187
+ end
188
+ end
189
+ end
190
+ end
@@ -84,9 +84,12 @@ module Railwatch
84
84
 
85
85
  # Puma is loaded in every process of an app that bundles it, so a Solid
86
86
  # Queue worker is recognised first, by how it was started (bin/jobs or
87
- # `rake solid_queue:start`).
87
+ # `rake solid_queue:start`) or by the procline Solid Queue gives every
88
+ # process it forks ("solid-queue-worker(1.7.0): ..."), which replaces
89
+ # $PROGRAM_NAME after boot and would otherwise turn the supervisor,
90
+ # dispatcher, and scheduler into "web" on every health sample.
88
91
  def role
89
- if defined?(::SolidQueue) && ($PROGRAM_NAME.include?("jobs") || ARGV.first.to_s.start_with?("solid_queue:")) then "worker"
92
+ if defined?(::SolidQueue) && ($PROGRAM_NAME.include?("jobs") || $PROGRAM_NAME.start_with?("solid-queue-") || ARGV.first.to_s.start_with?("solid_queue:")) then "worker"
90
93
  elsif defined?(::Rails::Console) then "console"
91
94
  elsif $PROGRAM_NAME.end_with?("rake") then "command"
92
95
  elsif defined?(::Puma) then "web"
@@ -12,6 +12,7 @@ require "railwatch/subscribers/storage"
12
12
  require "railwatch/subscribers/views"
13
13
  require "railwatch/subscribers/logs"
14
14
  require "railwatch/subscribers/jobs"
15
+ require "railwatch/subscribers/llm"
15
16
  require "railwatch/subscribers/deprecations"
16
17
  require "railwatch/subscribers/users"
17
18
  require "railwatch/subscribers/process_info"
@@ -19,7 +20,7 @@ require "railwatch/subscribers/process_info"
19
20
  module Railwatch
20
21
  module Subscribers
21
22
  ALL = [ Requests, Queries, Exceptions, Cache, Mail, Broadcasts, Notifications,
22
- Storage, Views, Logs, Jobs, Deprecations, Users, ProcessInfo ].freeze
23
+ Storage, Views, Logs, Jobs, Deprecations, Users, ProcessInfo, Llm ].freeze
23
24
 
24
25
  module_function
25
26
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Railwatch
4
- VERSION = "0.1.1"
4
+ VERSION = "0.1.3"
5
5
  end
data/lib/railwatch.rb CHANGED
@@ -496,7 +496,7 @@ module Railwatch
496
496
  query: :queries, n_plus_one: :queries, transaction: :transactions, cache_event: :cache_events,
497
497
  mail: :mail, broadcast: :broadcasts, notification: :notifications, outgoing_request: :outgoing_requests,
498
498
  storage_op: :storage_ops, view_render: :view_renders, log: :logs, deprecation: :deprecations,
499
- session: :sessions
499
+ session: :sessions, llm_call: :llm_calls
500
500
  }.freeze
501
501
 
502
502
  def type_plural(type)
metadata CHANGED
@@ -1,13 +1,13 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: railwatch
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.1.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Cole Robertson
8
8
  bindir: bin
9
9
  cert_chain: []
10
- date: 2026-09-14 00:00:00.000000000 Z
10
+ date: 2026-09-15 00:00:00.000000000 Z
11
11
  dependencies:
12
12
  - !ruby/object:Gem::Dependency
13
13
  name: rails
@@ -115,6 +115,7 @@ files:
115
115
  - lib/railwatch/subscribers/deprecations.rb
116
116
  - lib/railwatch/subscribers/exceptions.rb
117
117
  - lib/railwatch/subscribers/jobs.rb
118
+ - lib/railwatch/subscribers/llm.rb
118
119
  - lib/railwatch/subscribers/logs.rb
119
120
  - lib/railwatch/subscribers/mail.rb
120
121
  - lib/railwatch/subscribers/notifications.rb