newrelic_rpm 9.7.1 → 9.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6bdcea3ab1fea41b672c672c931a6005966bef1500709deb9d85681912cf7d33
-  data.tar.gz: 7d2bcd8db6e959d909e26dd4c3d8e8a99c7cec6572204a1c0283ce65521a5d6a
+  metadata.gz: fac159d955f3aaf1b926d68335c05f8bbea9fa3089bed68bee11f9bb50243361
+  data.tar.gz: 783c82413e45080741cbc9fd9fe0488b59cede5b64d359af46cf524d871b7763
 SHA512:
-  metadata.gz: '053846613ea0a66e691e15cd2159c3ada613f3f66dcfc9232f723569a54d3659f84b8b16ed178e7eb46691a36dc182addaf187370b15862e3f05d3e9335f50b3'
-  data.tar.gz: 588be3aa5b6e29b2cb1c79e89d4b3bd4af5fc5e41e819f1916c52107aca1ba41f8ca9d573ccf7d8def38d9fb166a2cd73ae2bf9733de2bc03472cd517cdf1801
+  metadata.gz: ae3deb3425be91f81d30fd76f6af95ec2ff573f2941e17474cbc96cadd70baa7eda534fcb8886ed039b71eba202d81ef9098600e307dcdf27919f84612eea15d
+  data.tar.gz: 4000a2bd73ea4eee6853878630d777226c39af0b46aafa8946bc167495728dc754136731162e660e6fbc895d2df1ca12709e3d2e60be4df8ae5f09035e072cfe

data/CHANGELOG.md

@@ -1,5 +1,35 @@
 # New Relic Ruby Agent Release Notes
 
+## v9.8.0
+
+Version 9.8.0 introduces instrumentation for ruby-openai, adds the option to store tracer state on the thread-level, hardens the browser agent insertion logic to better proactively anticipate errors, and prevents excpetions from being raised in the Active Support Broadcast logger instrumentation.
+
+- **Feature: Add instrumentation for ruby-openai**
+
+  Instrumentation has been added for the [ruby-openai](https://github.com/alexrudall/ruby-openai) gem, supporting versions 3.4.0 and higher [(PR#2442)](https://github.com/newrelic/newrelic-ruby-agent/pull/2442). While ruby-openai instrumentation is enabled by default, the configuration option `ai_monitoring.enabled` is disabled by default and controls all AI monitoring. `ai_monitoring.enabled` must be set to `true` in order to receive ruby-openai instrumentation. High-Security Mode must be disabled in order to receive AI monitoring.
+
+  Calls to embedding and chat completion endpoints are automatically traced. These events can be enhanced with the introduction of two new APIs. Custom attributes can also be added to LLM events using the API `NewRelic::Agent.add_custom_attributes`, but they must be prefixed with `llm.`. For example, `NewRelic::Agent.add_custom_attributes({'llm.user_id': user_id})`.
+
+- **Feature: Add AI monitoring APIs**
+
+  This version introduces two new APIs that allow users to record additional information on LLM events:
+  * `NewRelic::Agent.record_llm_feedback_event` - Records user feedback events.
+  * `NewRelic::Agent.set_llm_token_count_callback` - Sets a callback proc for calculating `token_count` attributes for embedding and chat completion message events.
+  
+  Visit [RubyDoc](https://rubydoc.info/github/newrelic/newrelic-ruby-agent/) for more information on each of these APIs.
+
+- **Feature: Store tracer state on thread-level**
+
+  A new configuration option, `thread_local_tracer_state`, stores New Relic's tracer state on the thread-level, as opposed to the default fiber-level storage. This configuration is turned off by default. Our thanks go to community member [@markiz](https://github.com/markiz) who contributed the idea, code, configuration option, and tests for this new feature! [PR#2475](https://github.com/newrelic/newrelic-ruby-agent/pull/2475).
+
+- **Bugfix: Harden the browser agent insertion logic**
+
+  With [Issue#2462](https://github.com/newrelic/newrelic-ruby-agent/issues/2462), community member [@miry](https://github.com/miry) explained that it was possible for an HTTP response headers hash to have symbols for values. Not only would these symbols prevent the inclusion of the New Relic browser agent tag in the response body, but more importantly they would cause an exception that would bubble up to the monitored web application itself. With [PR#2465](https://github.com/newrelic/newrelic-ruby-agent/pull/2465) symbol based values are now supported and all other potential future exceptions are now handled. Additionally, the refactor to support symbols has been shown through benchmarking to give the processing of string and mixed type hashes a slight speed boost too.
+
+- **Bugfix: Prevent Exception in Active Support Broadcast logger instrumentation**
+
+  Previously, in certain situations the agent could cause an exception to be raised when attempting to interact with a broadcast log event. This has been fixed. Thanks to [@nathan-appere](https://github.com/nathan-appere) for reporting this issue and providing a fix! [PR#2510](https://github.com/newrelic/newrelic-ruby-agent/pull/2510)
+
 
 ## v9.7.1
 

data/README.md

@@ -119,7 +119,7 @@ If you have any questions, or to execute our corporate CLA (required if your con
 
 As noted in our [security policy](https://github.com/newrelic/newrelic-ruby-agent/security/policy), New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.
 
-If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through [HackerOne](https://hackerone.com/newrelic).
+If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through [our bug bounty program](https://docs.newrelic.com/docs/security/security-privacy/information-security/report-security-vulnerabilities/).
 
 If you would like to contribute to this project, please review [these guidelines](https://github.com/newrelic/newrelic-ruby-agent/blob/main/CONTRIBUTING.md).
 

data/lib/new_relic/agent/configuration/default_source.rb

@@ -360,6 +360,26 @@ module NewRelic
               - a.third.event
           DESCRIPTION
         },
+        :'ai_monitoring.enabled' => {
+          :default => false,
+          :public => true,
+          :type => Boolean,
+          :allowed_from_server => false,
+          :description => 'If `false`, all LLM instrumentation (OpenAI only for now) will be disabled and no metrics, events, or spans will be sent. AI Monitoring is automatically disabled if `high_security` mode is enabled.'
+        },
+        :'ai_monitoring.record_content.enabled' => {
+          :default => true,
+          :public => true,
+          :type => Boolean,
+          :allowed_from_server => false,
+          :description => <<~DESCRIPTION
+            If `false`, LLM instrumentation (OpenAI only for now) will not capture input and output content on specific LLM events.
+
+            The excluded attributes include:
+              * `content` from LlmChatCompletionMessage events
+              * `input` from LlmEmbedding events
+          DESCRIPTION
+        },
         # this is only set via server side config
         :apdex_t => {
           :default => 0.5,
@@ -554,6 +574,13 @@ module NewRelic
           :allowed_from_server => false,
           :description => 'When set to `true`, forces a synchronous connection to the New Relic [collector](/docs/using-new-relic/welcome-new-relic/get-started/glossary/#collector) during application startup. For very short-lived processes, this helps ensure the New Relic agent has time to report.'
         },
+        :thread_local_tracer_state => {
+          :default => false,
+          :public => true,
+          :type => Boolean,
+          :allowed_from_server => false,
+          :description => 'If `true`, tracer state storage is thread-local, otherwise, fiber-local'
+        },
         :timeout => {
           :default => 2 * 60, # 2 minutes
           :public => true,
@@ -720,7 +747,7 @@ module NewRelic
           :public => true,
           :type => Integer,
           :allowed_from_server => false,
-          :description => 'Defines the maximum number of frames in an error backtrace. Backtraces over this amount are truncated at the beginning and end.'
+          :description => 'Defines the maximum number of frames in an error backtrace. Backtraces over this amount are truncated in the middle, preserving the beginning and the end of the stack trace.'
         },
         :'error_collector.max_event_samples_stored' => {
           :default => 100,
@@ -1570,6 +1597,15 @@ module NewRelic
           :allowed_from_server => false,
           :description => 'Controls auto-instrumentation of `Net::HTTP` at start-up. May be one of: `auto`, `prepend`, `chain`, `disabled`.'
         },
+        :'instrumentation.ruby_openai' => {
+          :default => 'auto',
+          :documentation_default => 'auto',
+          :public => true,
+          :type => String,
+          :dynamic_name => true,
+          :allowed_from_server => false,
+          :description => 'Controls auto-instrumentation of the ruby-openai gem at start-up. May be one of: `auto`, `prepend`, `chain`, `disabled`.'
+        },
         :'instrumentation.puma_rack' => {
           :default => value_of(:'instrumentation.rack'),
           :documentation_default => 'auto',

data/lib/new_relic/agent/configuration/high_security_source.rb

@@ -19,6 +19,7 @@ module NewRelic
             :'elasticsearch.obfuscate_queries' => true,
             :'transaction_tracer.record_redis_arguments' => false,
 
+            :'ai_monitoring.enabled' => false,
             :'custom_insights_events.enabled' => false,
             :'strip_exception_messages.enabled' => true
           })

data/lib/new_relic/agent/configuration/manager.rb

@@ -34,6 +34,7 @@ module NewRelic
         def initialize
           reset_to_defaults
           @callbacks = Hash.new { |hash, key| hash[key] = [] }
+          @lock = Mutex.new
         end
 
         def add_config_for_testing(source, level = 0)
@@ -364,9 +365,11 @@ module NewRelic
         def reset_cache
           return new_cache unless defined?(@cache) && @cache
 
-          preserved = @cache.dup.select { |_k, v| DEPENDENCY_DETECTION_VALUES.include?(v) }
-          new_cache
-          preserved.each { |k, v| @cache[k] = v }
+          @lock.synchronize do
+            preserved = @cache.dup.select { |_k, v| DEPENDENCY_DETECTION_VALUES.include?(v) }
+            new_cache
+            preserved.each { |k, v| @cache[k] = v }
+          end
 
           @cache
         end

data/lib/new_relic/agent/configuration/security_policy_source.rb

@@ -7,6 +7,8 @@ require 'new_relic/agent/configuration/dotted_hash'
 module NewRelic
   module Agent
     module Configuration
+      # The Language Security Policy Source gives customers the ability to
+      # configure high security mode settings.
       class SecurityPolicySource < DottedHash
         class << self
           def enabled?(option)
@@ -147,6 +149,15 @@ module NewRelic
               permitted_fn: nil
             }
           ],
+          'ai_monitoring' => [
+            {
+              option: :'ai_monitoring.enabled',
+              supported: true,
+              enabled_fn: method(:enabled?),
+              disabled_value: false,
+              permitted_fn: nil
+            }
+          ],
           'allow_raw_exception_messages' => [
             {
               option: :'strip_exception_messages.enabled',

data/lib/new_relic/agent/custom_event_aggregator.rb

@@ -52,11 +52,11 @@ module NewRelic
           {TYPE => type,
            TIMESTAMP => Process.clock_gettime(Process::CLOCK_REALTIME).to_i,
            PRIORITY => priority},
-          create_custom_event_attributes(attributes)
+          create_custom_event_attributes(type, attributes)
         ]
       end
 
-      def create_custom_event_attributes(attributes)
+      def create_custom_event_attributes(type, attributes)
         result = AttributeProcessing.flatten_and_coerce(attributes)
 
         if result.size > MAX_ATTRIBUTE_COUNT
@@ -70,9 +70,9 @@ module NewRelic
             key = key[0, MAX_NAME_SIZE]
           end
 
-          # value is limited to 4095
+          # value is limited to 4095 except for LLM content-related events
           if val.is_a?(String) && val.length > MAX_ATTRIBUTE_SIZE
-            val = val[0, MAX_ATTRIBUTE_SIZE]
+            val = val[0, MAX_ATTRIBUTE_SIZE] unless NewRelic::Agent::LLM.exempt_event_attribute?(type, key)
           end
 
           new_result[key] = val

data/lib/new_relic/agent/error_collector.rb

@@ -216,6 +216,8 @@ module NewRelic
       def notice_segment_error(segment, exception, options = {})
         return if skip_notice_error?(exception)
 
+        options.merge!(segment.llm_event.error_attributes(exception)) if segment.llm_event
+
         segment.set_noticed_error(create_noticed_error(exception, options))
         exception
       rescue => e

data/lib/new_relic/agent/instrumentation/active_support_broadcast_logger/instrumentation.rb

@@ -5,9 +5,13 @@
 module NewRelic::Agent::Instrumentation
   module ActiveSupportBroadcastLogger
     def record_one_broadcast_with_new_relic(*args)
-      broadcasts[1..-1].each { |broadcasted_logger| broadcasted_logger.instance_variable_set(:@skip_instrumenting, true) }
-      yield
-      broadcasts.each { |broadcasted_logger| broadcasted_logger.instance_variable_set(:@skip_instrumenting, false) }
+      if broadcasts && broadcasts[1..-1]
+        broadcasts[1..-1].each { |broadcasted_logger| broadcasted_logger.instance_variable_set(:@skip_instrumenting, true) }
+        yield
+        broadcasts.each { |broadcasted_logger| broadcasted_logger.instance_variable_set(:@skip_instrumenting, false) }
+      else
+        NewRelic::Agent.logger.error('Error recording broadcasted logger')
+      end
     end
   end
 end

data/lib/new_relic/agent/instrumentation/concurrent_ruby.rb

@@ -11,6 +11,7 @@ DependencyDetection.defer do
 
   depends_on do
     defined?(Concurrent) &&
+      defined?(Concurrent::VERSION) &&
       Gem::Version.new(Concurrent::VERSION) >= Gem::Version.new('1.1.5')
   end
 

data/lib/new_relic/agent/instrumentation/net_http/instrumentation.rb

@@ -32,7 +32,13 @@ module NewRelic
             end
 
             wrapped_response = NewRelic::Agent::HTTPClients::NetHTTPResponse.new(response)
+
+            if NewRelic::Agent::LLM.openai_parent?(segment)
+              NewRelic::Agent::LLM.populate_openai_response_headers(wrapped_response, segment.parent)
+            end
+
             segment.process_response_headers(wrapped_response)
+
             response
           ensure
             segment&.finish

data/lib/new_relic/agent/instrumentation/ruby_openai/chain.rb

@@ -0,0 +1,36 @@
+# This file is distributed under New Relic's license terms.
+# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
+# frozen_string_literal: true
+
+module NewRelic::Agent::Instrumentation
+  module OpenAI::Chain
+    def self.instrument!
+      ::OpenAI::Client.class_eval do
+        include NewRelic::Agent::Instrumentation::OpenAI
+
+        alias_method(:json_post_without_new_relic, :json_post)
+
+        # In versions 4.0.0+ json_post is an instance method
+        # defined in the OpenAI::HTTP module, included by the
+        # OpenAI::Client class
+        def json_post(**kwargs)
+          json_post_with_new_relic(**kwargs) do
+            json_post_without_new_relic(**kwargs)
+          end
+        end
+
+        # In versions below 4.0.0 json_post is a class method
+        # on OpenAI::Client
+        class << self
+          alias_method(:json_post_without_new_relic, :json_post)
+
+          def json_post(**kwargs)
+            json_post_with_new_relic(**kwargs) do
+              json_post_without_new_relic(**kwargs)
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/new_relic/agent/instrumentation/ruby_openai/instrumentation.rb

@@ -0,0 +1,197 @@
+# This file is distributed under New Relic's license terms.
+# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
+# frozen_string_literal: true
+
+module NewRelic::Agent::Instrumentation
+  module OpenAI
+    VENDOR = 'openAI' # AIM expects this capitalization style for the UI
+    INSTRUMENTATION_NAME = NewRelic::Agent.base_name(name)
+    EMBEDDINGS_PATH = '/embeddings'
+    CHAT_COMPLETIONS_PATH = '/chat/completions'
+    EMBEDDINGS_SEGMENT_NAME = 'Llm/embedding/OpenAI/embeddings'
+    CHAT_COMPLETIONS_SEGMENT_NAME = 'Llm/completion/OpenAI/chat'
+
+    def json_post_with_new_relic(path:, parameters:)
+      return yield unless path == EMBEDDINGS_PATH || path == CHAT_COMPLETIONS_PATH
+
+      NewRelic::Agent.record_instrumentation_invocation(INSTRUMENTATION_NAME)
+      NewRelic::Agent::Llm::LlmEvent.set_llm_agent_attribute_on_transaction
+      record_openai_metric
+
+      if path == EMBEDDINGS_PATH
+        embeddings_instrumentation(parameters) { yield }
+      elsif path == CHAT_COMPLETIONS_PATH
+        chat_completions_instrumentation(parameters) { yield }
+      end
+    end
+
+    private
+
+    def embeddings_instrumentation(parameters)
+      segment = NewRelic::Agent::Tracer.start_segment(name: EMBEDDINGS_SEGMENT_NAME)
+      event = create_embeddings_event(parameters)
+      segment.llm_event = event
+      begin
+        response = NewRelic::Agent::Tracer.capture_segment_error(segment) { yield }
+        # TODO: Remove !response.include?('error') when we drop support for versions below 4.0.0
+        add_embeddings_response_params(response, event) if response && !response.include?('error')
+
+        response
+      ensure
+        finish(segment, event)
+      end
+    end
+
+    def chat_completions_instrumentation(parameters)
+      segment = NewRelic::Agent::Tracer.start_segment(name: CHAT_COMPLETIONS_SEGMENT_NAME)
+      event = create_chat_completion_summary(parameters)
+      segment.llm_event = event
+      messages = create_chat_completion_messages(parameters, event.id)
+
+      begin
+        response = NewRelic::Agent::Tracer.capture_segment_error(segment) { yield }
+        # TODO: Remove !response.include?('error') when we drop support for versions below 4.0.0
+        if response && !response.include?('error')
+          add_chat_completion_response_params(parameters, response, event)
+          messages = update_chat_completion_messages(messages, response, event)
+        end
+
+        response
+      ensure
+        finish(segment, event)
+        messages&.each { |m| m.record }
+      end
+    end
+
+    def create_chat_completion_summary(parameters)
+      NewRelic::Agent::Llm::ChatCompletionSummary.new(
+        vendor: VENDOR,
+        request_max_tokens: (parameters[:max_tokens] || parameters['max_tokens'])&.to_i,
+        request_model: parameters[:model] || parameters['model'],
+        temperature: (parameters[:temperature] || parameters['temperature'])&.to_f,
+        metadata: llm_custom_attributes
+      )
+    end
+
+    def create_embeddings_event(parameters)
+      event = NewRelic::Agent::Llm::Embedding.new(
+        vendor: VENDOR,
+        request_model: parameters[:model] || parameters['model'],
+        metadata: llm_custom_attributes
+      )
+      add_input(event, (parameters[:input] || parameters['input']))
+
+      event
+    end
+
+    def add_chat_completion_response_params(parameters, response, event)
+      event.response_number_of_messages = (parameters[:messages] || parameters['messages']).size + response['choices'].size
+      # The response hash always returns keys as strings, so we don't need to run an || check here
+      event.response_model = response['model']
+      event.response_choices_finish_reason = response['choices'][0]['finish_reason']
+    end
+
+    def add_embeddings_response_params(response, event)
+      event.response_model = response['model']
+      event.token_count = calculate_token_count(event.request_model, event.input)
+    end
+
+    def create_chat_completion_messages(parameters, summary_id)
+      (parameters[:messages] || parameters['messages']).map.with_index do |message, index|
+        msg = NewRelic::Agent::Llm::ChatCompletionMessage.new(
+          role: message[:role] || message['role'],
+          sequence: index,
+          completion_id: summary_id,
+          vendor: VENDOR
+        )
+        add_content(msg, (message[:content] || message['content']))
+
+        msg
+      end
+    end
+
+    def create_chat_completion_response_messages(response, sequence_origin, summary_id)
+      response['choices'].map.with_index(sequence_origin) do |choice, index|
+        msg = NewRelic::Agent::Llm::ChatCompletionMessage.new(
+          role: choice['message']['role'],
+          sequence: index,
+          completion_id: summary_id,
+          vendor: VENDOR,
+          is_response: true
+        )
+        add_content(msg, choice['message']['content'])
+
+        msg
+      end
+    end
+
+    def update_chat_completion_messages(messages, response, summary)
+      messages += create_chat_completion_response_messages(response, messages.size, summary.id)
+      response_id = response['id'] || NewRelic::Agent::GuidGenerator.generate_guid
+
+      messages.each do |message|
+        message.id = "#{response_id}-#{message.sequence}"
+        message.request_id = summary.request_id
+        message.response_model = response['model']
+        message.metadata = llm_custom_attributes
+
+        model = message.is_response ? message.response_model : summary.request_model
+
+        message.token_count = calculate_token_count(model, message.content)
+      end
+    end
+
+    def calculate_token_count(model, content)
+      return unless NewRelic::Agent.llm_token_count_callback
+
+      begin
+        count = NewRelic::Agent.llm_token_count_callback.call({model: model, content: content})
+      rescue => e
+        NewRelic::Agent.logger.warn("Error calculating token count using the provided proc. Error: #{e}'")
+      end
+
+      count if count.is_a?(Integer) && count > 0
+    end
+
+    def record_content_enabled?
+      NewRelic::Agent.config[:'ai_monitoring.record_content.enabled']
+    end
+
+    def add_content(message, content)
+      message.content = content if record_content_enabled?
+    end
+
+    def add_input(event, input)
+      event.input = input if record_content_enabled?
+    end
+
+    def llm_custom_attributes
+      NewRelic::Agent::Tracer.current_transaction&.attributes&.custom_attributes&.select { |k| k.to_s.match(/llm.*/) }
+    end
+
+    def record_openai_metric
+      NewRelic::Agent.record_metric(nr_supportability_metric, 0.0)
+    end
+
+    def segment_noticed_error?(segment)
+      segment&.noticed_error
+    end
+
+    def nr_supportability_metric
+      @nr_supportability_metric ||= "Supportability/Ruby/ML/OpenAI/#{::OpenAI::VERSION}"
+    end
+
+    def finish(segment, event)
+      segment&.finish
+
+      return unless event
+
+      if segment
+        event.error = true if segment_noticed_error?(segment)
+        event.duration = segment.duration
+      end
+
+      event.record
+    end
+  end
+end

data/lib/new_relic/agent/instrumentation/ruby_openai/prepend.rb

@@ -0,0 +1,20 @@
+# This file is distributed under New Relic's license terms.
+# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
+# frozen_string_literal: true
+
+module NewRelic::Agent::Instrumentation
+  module OpenAI::Prepend
+    include NewRelic::Agent::Instrumentation::OpenAI
+
+    # In versions 4.0.0+ json_post is an instance method defined in the
+    # OpenAI::HTTP module, included by the OpenAI::Client class.
+    #
+    # In versions below 4.0.0 json_post is a class method on OpenAI::Client.
+    #
+    # Dependency detection will apply the instrumentation to the correct scope,
+    # so we don't need to change the code here.
+    def json_post(**kwargs)
+      json_post_with_new_relic(**kwargs) { super }
+    end
+  end
+end

data/lib/new_relic/agent/instrumentation/ruby_openai.rb

@@ -0,0 +1,35 @@
+# This file is distributed under New Relic's license terms.
+# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
+# frozen_string_literal: true
+
+require_relative 'ruby_openai/instrumentation'
+require_relative 'ruby_openai/chain'
+require_relative 'ruby_openai/prepend'
+
+DependencyDetection.defer do
+  named :'ruby_openai'
+
+  depends_on do
+    NewRelic::Agent.config[:'ai_monitoring.enabled'] &&
+      defined?(OpenAI) && defined?(OpenAI::Client) &&
+      Gem::Version.new(OpenAI::VERSION) >= Gem::Version.new('3.4.0')
+  end
+
+  executes do
+    if use_prepend?
+      # TODO: Remove condition when we drop support for versions below 5.0.0
+      if Gem::Version.new(OpenAI::VERSION) >= Gem::Version.new('5.0.0')
+        prepend_instrument OpenAI::Client,
+          NewRelic::Agent::Instrumentation::OpenAI::Prepend,
+          NewRelic::Agent::Instrumentation::OpenAI::VENDOR
+      else
+        prepend_instrument OpenAI::Client.singleton_class,
+          NewRelic::Agent::Instrumentation::OpenAI::Prepend,
+          NewRelic::Agent::Instrumentation::OpenAI::VENDOR
+      end
+    else
+      chain_instrument NewRelic::Agent::Instrumentation::OpenAI::Chain,
+        NewRelic::Agent::Instrumentation::OpenAI::VENDOR
+    end
+  end
+end

data/lib/new_relic/agent/llm/chat_completion_message.rb

@@ -0,0 +1,25 @@
+# This file is distributed under New Relic's license terms.
+# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
+# frozen_string_literal: true
+
+module NewRelic
+  module Agent
+    module Llm
+      class ChatCompletionMessage < LlmEvent
+        ATTRIBUTES = %i[content role sequence completion_id token_count
+          is_response]
+        EVENT_NAME = 'LlmChatCompletionMessage'
+
+        attr_accessor(*ATTRIBUTES)
+
+        def attributes
+          LlmEvent::ATTRIBUTES + ATTRIBUTES
+        end
+
+        def event_name
+          EVENT_NAME
+        end
+      end
+    end
+  end
+end

data/lib/new_relic/agent/llm/chat_completion_summary.rb

@@ -0,0 +1,66 @@
+# This file is distributed under New Relic's license terms.
+# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
+# frozen_string_literal: true
+
+require_relative 'response_headers'
+
+module NewRelic
+  module Agent
+    module Llm
+      class ChatCompletionSummary < LlmEvent
+        include ResponseHeaders
+
+        ATTRIBUTES = %i[request_max_tokens response_number_of_messages
+          request_model response_choices_finish_reason request_temperature
+          duration error]
+        ATTRIBUTE_NAME_EXCEPTIONS = {
+          response_number_of_messages: 'response.number_of_messages',
+          request_model: 'request.model',
+          response_choices_finish_reason: 'response.choices.finish_reason',
+          request_temperature: 'request.temperature'
+        }
+        ERROR_COMPLETION_ID = 'completion_id'
+        EVENT_NAME = 'LlmChatCompletionSummary'
+
+        attr_accessor(*ATTRIBUTES)
+
+        def attributes
+          LlmEvent::ATTRIBUTES + ResponseHeaders::ATTRIBUTES + ATTRIBUTES
+        end
+
+        def attribute_name_exceptions
+          # TODO: OLD RUBIES < 2.6
+          # Hash#merge accepts multiple arguments in 2.6
+          # Remove condition once support for Ruby <2.6 is dropped
+          if RUBY_VERSION >= '2.6.0'
+            LlmEvent::ATTRIBUTE_NAME_EXCEPTIONS.merge(ResponseHeaders::ATTRIBUTE_NAME_EXCEPTIONS, ATTRIBUTE_NAME_EXCEPTIONS)
+          else
+            LlmEvent::ATTRIBUTE_NAME_EXCEPTIONS.merge(ResponseHeaders::ATTRIBUTE_NAME_EXCEPTIONS).merge(ATTRIBUTE_NAME_EXCEPTIONS)
+          end
+        end
+
+        def event_name
+          EVENT_NAME
+        end
+
+        def error_attributes(exception)
+          attrs = {ERROR_COMPLETION_ID => id}
+
+          error_attributes_from_response(exception, attrs)
+        end
+
+        private
+
+        def error_attributes_from_response(exception, attrs)
+          return attrs unless exception.respond_to?(:response)
+
+          attrs[ERROR_ATTRIBUTE_STATUS_CODE] = exception.response.dig(:status)
+          attrs[ERROR_ATTRIBUTE_CODE] = exception.response.dig(:body, ERROR_STRING, CODE_STRING)
+          attrs[ERROR_ATTRIBUTE_PARAM] = exception.response.dig(:body, ERROR_STRING, PARAM_STRING)
+
+          attrs
+        end
+      end
+    end
+  end
+end