nitro_intelligence 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: aa22292af25ca70dd3a19ba62830fa25606a84517af6f7bb79906ceea4d0e058
+  data.tar.gz: 07c68668426d3feb046e633203e617ede8f9ef807a3f60341c347fbb0c05b17f
+SHA512:
+  metadata.gz: 02d7bcd08eb45f6dc14c0b4a5e5f91e0edac92369435721e838a0e623a5a26b4af33b0f851c1d02de54ca6d80a864d0b8904e9a78612e9d14aa642e8993865ba
+  data.tar.gz: e6c1bfc0896d5d11cc017e23aef1b617c3f83182c09fe8a6414202d66dfbdc5d2e13fe73bf3b5ef54e928d5f1cf944ad001781397bab7a8a5f4867e4a152cc1c

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec

data/docs/README.md

@@ -0,0 +1,247 @@
+# Nitro Intelligence
+
+The entrypoint to everything AI.
+
+This component aims to consolidate and standardize AI features implemented in the host application.
+
+## Configuration
+
+`NitroIntelligence` is configured via an initializer in the host application. All external dependencies
+(e.g. inference API credentials, observability settings) must be injected at boot time:
+
+```ruby
+NitroIntelligence.configure do |config|
+  # Standard Rails integrations
+  config.logger         = Rails.logger           # Logger instance
+  config.environment    = Rails.env              # e.g. "production", "test"
+  config.cache_provider = Rails.cache            # ActiveSupport cache store
+
+  # Inference (LLM) settings
+  config.inference_api_key  = "..."              # API key for the inference service
+  config.inference_base_url = "https://..."      # Base URL for the inference service
+
+  # Observability (Langfuse) settings
+  config.observability_base_url = "https://..."  # Base URL for the observability service
+  config.observability_projects = [              # Array of project credential hashes
+    {
+      "slug"       => "my-feature-project",
+      "id"         => "project-id",
+      "public_key" => "pk-...",
+      "secret_key" => "sk-...",
+    },
+  ]
+
+  # Agent server settings (optional)
+  config.agent_server_config = {}                # Hash of AgentServer keyword arguments
+end
+```
+
+### Configuration Keys
+
+| Key | Type | Default | Description |
+|---|---|---|---|
+| `logger` | `Logger` | `Logger.new($stdout)` | Logger used for diagnostic output |
+| `environment` | `String` | `"test"` | Runtime environment name |
+| `cache_provider` | cache store | `NullCache` | ActiveSupport-compatible cache store |
+| `inference_api_key` | `String` | `""` | API key for the LLM inference service |
+| `inference_base_url` | `String` | `""` | Base URL for the LLM inference service |
+| `observability_base_url` | `String` | `""` | Base URL for the Langfuse observability service |
+| `observability_projects` | `Array<Hash>` | `[]` | Langfuse project credentials (slug, id, public_key, secret_key) |
+| `agent_server_config` | `Hash` | `{}` | Credentials for `AgentServer.new`. Expected keys: `base_url` (String) — HTTP base URL of the agent server; `api_key` (String) — bearer token; `user_id` (String, default: `"default-user"`) — caller identity |
+
+## Basic Usage
+
+### OpenAI API/LLM Requests
+
+A simple LLM call can be invoked as follows:
+
+```ruby
+client = NitroIntelligence::Client.new
+client.chat(message: "Why is the sky blue?")
+content = result.choices.first&.message&.content
+```
+
+This component handles setting defaults for most things, such as model, host, keys, etc.
+
+You may also use [`openai-ruby`](https://github.com/openai/openai-ruby) compatible syntax with this wrapper by passing the parameters keyword in. For example:
+
+```ruby
+client = NitroIntelligence::Client.new
+client.chat(parameters: { model: "meta-llama/Llama-3.1-8B-Instruct", messages: [{ role: "user", content: "Why is the sky blue?" }]})
+```
+
+
+#### Providing Parameters
+
+Parameters such as 'max_tokens' and 'temperature' can be passed in under the `parameters` key.
+
+```ruby
+client = NitroIntelligence::Client.new
+client.chat(parameters: { model: "meta-llama/Llama-3.1-8B-Instruct", max_tokens: 1000, temperature: 0.7, messages: [{ role: "user", content: "Why is the sky blue?" }]})
+```
+
+For a full list of supported parameters, see the [API reference here](https://developers.openai.com/api/reference/resources/completions/methods/create).
+
+### Image Editing and Generation
+
+Nitro Intelligence can be used for image editing and generation
+
+Basic examples of usage:
+
+#### Image Generation
+
+```ruby
+client = NitroIntelligence::Client.new
+result = client.generate_image(message: "Create an image of a bear installing a window.")
+```
+
+`result` is a `NitroIntelligence::ImageGeneration` object, and the generated image can be accessed via `result.generated_image` which returns a `NitroIntelligence::Image`.
+
+You may write the file to disk:
+
+```ruby
+File.binwrite("my_generated_image.#{result.generated_image.file_extension}", result.generated_image.byte_string)
+```
+
+#### Image Editing and Uploading Reference Images
+
+To edit an image, provide your source image as a byte string, along with any references you would like to include.
+
+```ruby
+client = NitroIntelligence::Client.new
+house = File.binread("./house.jpg")
+siding = File.binread("./siding.png")
+result = client.generate_image(message: "Replace the siding in the image of the house with the new siding I have provided.", target_image: house, reference_images: [siding])
+```
+
+#### Using Prompts
+
+See Observability[## Observability] for more details. Basic usage looks like:
+
+```ruby
+client = NitroIntelligence::Client.new(observability_project_slug: "sample-project-slug")
+house = File.binread("./house.jpg")
+siding = File.binread("./siding.png")
+result = client.generate_image(target_image: house, reference_images: [siding], parameters: {prompt_name: "Sample Prompt Name"})
+```
+
+#### Image Configuration
+
+You can specify parameters such as model to use, aspect ratio and resolution via the `parameters` key:
+
+```ruby
+client = NitroIntelligence::Client.new
+result = client.generate_image(message: "Create an image of a bear installing a window.", parameters: {aspect_ratio: "4:3", resolution: "512"})
+```
+
+## Observability
+
+### Setup
+
+If your feature is setup for observability, simply pass the observability project and desired prompt name in when invoking requests. It is also preferable to pass in a source and any other useful metadata for later debugging. For example:
+
+```ruby
+client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
+client.chat(
+  message: "Why is the sky blue?",
+  parameters: {
+    prompt_name: "My Prompt",
+    # prompt_label: "debug",
+    # prompt_version: "v2",
+    metadata: {
+      source: self.class,
+    },
+  }
+)
+```
+
+If no `prompt_label` or `prompt_version` is provided, the 'production' prompt is used by default.
+
+### Custom Trace Names
+
+To provide custom trace names to the observability platform, you can pass 'trace_name' in parameters. Example:
+
+```ruby
+client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
+client.chat(
+  message: "Why is the sky blue?",
+  parameters: {
+    prompt_name: "My Prompt",
+    metadata: {
+      source: self.class,
+    },
+    trace_name: "custom-trace-name"
+  }
+)
+```
+
+### Prompt Variables and Config
+
+Prompts are often created with "variables". These variables can be supplied and compiled into the prompt. For example:
+
+```ruby
+client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
+client.chat(
+  message: "Where is the appointment?",
+  parameters: {
+    prompt_name: "My Prompt With Variables",
+    prompt_variables: {
+      appointment_id: "1234",
+    },
+    metadata: {
+      source: self.class,
+    },
+  }
+)
+```
+
+Prompts can be created with a "config" JSON object. This config stores structured data such as model parameters (like model name, temperature), function/tool parameters, or JSON schemas.
+
+The prompt config is merged into your chat request by default. The prompt config object is merged into the "parameters" hash, overriding any existing keys.
+
+Consider this prompt config:
+
+```json
+{
+  "model": "gpt-4o-mini"
+}
+```
+
+Invoking this request would result in "gpt-4o-mini" being used as the model, even if supplied manually:
+
+```ruby
+client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
+client.chat(
+  message: "Where is the appointment?",
+  parameters: {
+    model: "meta-llama/Llama-3.1-8B-Instruct", # Will not be used, will be overridden by config "gpt-4o-mini"
+    prompt_name: "My Prompt With Variables",
+    prompt_variables: {
+      appointment_id: "1234",
+    },
+    metadata: {
+      source: self.class,
+    },
+  }
+)
+```
+
+To disable the prompt config entirely, which may be useful for debugging/testing, you can supply the "prompt_config_disabled" keyword. For example:
+
+```ruby
+client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
+client.chat(
+  message: "Where is the appointment?",
+  parameters: {
+    model: "meta-llama/Llama-3.1-8B-Instruct", # This will now be used since "prompt_config_disabled" is true
+    prompt_name: "My Prompt With Variables",
+    prompt_variables: {
+      appointment_id: "1234",
+    },
+    prompt_config_disabled: true,
+    metadata: {
+      source: self.class,
+    },
+  }
+)
+```

data/lib/nitro_intelligence/agent_server.rb

@@ -0,0 +1,76 @@
+module NitroIntelligence
+  class AgentServer
+    class ConfigurationError < StandardError; end
+    class ThreadInitializationError < StandardError; end
+    class RunError < StandardError; end
+
+    attr_reader :base_url, :user_id
+
+    def initialize(base_url:, api_key:, user_id: "default-user")
+      raise ConfigurationError, "base_url is required" if base_url.blank?
+      raise ConfigurationError, "api_key is required" if api_key.blank?
+      raise ConfigurationError, "user_id is required" if user_id.blank?
+
+      @base_url = base_url
+      @api_key = api_key
+      @user_id = user_id
+    end
+
+    def await_run(thread_id:, assistant_id:, messages:, context: {})
+      raise RunError, "messages cannot be empty" if messages.blank?
+
+      *initial_state, last_message = messages
+      initial_state = [] if messages.size == 1
+      last_message = messages.first if messages.size == 1
+
+      initialize_thread_if_needed(thread_id:, initial_state:)
+      trigger_run(thread_id:, assistant_id:, context:, last_message:)
+    end
+
+  private
+
+    def initialize_thread_if_needed(thread_id:, initial_state:)
+      thread_response = post(
+        path: "/threads",
+        body: {
+          threadId: thread_id.to_s,
+          ifExists: "do_nothing",
+          initial_state: { messages: initial_state },
+          user_id:,
+        }
+      )
+
+      raise ThreadInitializationError, thread_response.body if thread_response.code != 200
+
+      thread_response
+    end
+
+    def trigger_run(thread_id:, assistant_id:, last_message:, context: {})
+      run_response = post(
+        path: "/threads/#{thread_id}/runs/wait",
+        body: {
+          assistant_id:,
+          context:,
+          input: {
+            messages: [last_message],
+          },
+        }
+      )
+
+      raise RunError, run_response.body if run_response.code != 200
+
+      run_response["messages"].last["content"]
+    end
+
+    def post(path:, body:)
+      HTTParty.post(
+        "#{base_url}#{path}",
+        headers: {
+          "Content-Type" => "application/json",
+          "Authorization" => "Bearer #{@api_key}",
+        },
+        body: body.to_json
+      )
+    end
+  end
+end

data/lib/nitro_intelligence/client.rb

@@ -0,0 +1,337 @@
+require "nitro_intelligence/trace"
+
+module NitroIntelligence
+  class Client
+    attr_accessor :client
+
+    def initialize(observability_project_slug: nil)
+      @inference_api_key = NitroIntelligence.config.inference_api_key
+      @inference_host = NitroIntelligence.config.inference_base_url
+      @observability_host = NitroIntelligence.config.observability_base_url
+      @observability_project_slug = observability_project_slug
+      @client = OpenAI::Client.new(
+        api_key: @inference_api_key,
+        base_url: @inference_host
+      )
+      @langfuse_client = NitroIntelligence.langfuse_clients[observability_project_slug]
+    end
+
+    def chat(message: "", parameters: {})
+      default_params = CUSTOM_PARAMS.index_with { |_param| nil }
+                                    .merge({
+                                             metadata: {},
+                                             messages: [],
+                                             model: NitroIntelligence.model_catalog.default_text_model.name,
+                                             observability_project_slug:,
+                                           })
+      parameters = default_params.merge(parameters)
+
+      parameters[:messages] = [{ role: "user", content: message }] if parameters[:messages].blank? && message.present?
+
+      return chat_with_tracing(parameters:) if observability_available?
+
+      client_chat(parameters:)
+    end
+
+    # We abstract the image generation for now because of usage
+    # across various apis: chat/completions, image/edits, image/generations
+    # Input images should be byte strings
+    # Returns NitroIntelligence::ImageGeneration
+    def generate_image(
+      message: "",
+      target_image: nil,
+      reference_images: [],
+      parameters: {}
+    )
+      image_generation = NitroIntelligence::ImageGeneration.new(message:, target_image:, reference_images:) do |config|
+        config.aspect_ratio = parameters[:aspect_ratio] if parameters.key?(:aspect_ratio)
+        config.model = parameters[:model] if parameters.key?(:model)
+        config.resolution = parameters[:resolution] if parameters.key?(:resolution)
+      end
+
+      default_params = CUSTOM_PARAMS.index_with { |_param| nil }
+                                    .merge({
+                                             image_generation:,
+                                             metadata: {},
+                                             messages: image_generation.messages,
+                                             model: image_generation.config.model,
+                                             observability_project_slug:,
+                                             extra_headers: {
+                                               "Prefer" => "wait",
+                                             },
+                                             request_options: {
+                                               extra_body: {
+                                                 image_config: {
+                                                   aspect_ratio: image_generation.config.aspect_ratio,
+                                                   image_size: image_generation.config.resolution,
+                                                 },
+                                               },
+                                             },
+                                           })
+      parameters = default_params.merge(parameters)
+
+      if observability_available?
+        chat_with_tracing(parameters:)
+      else
+        chat_completion = client_chat(parameters:)
+        image_generation.parse_file(chat_completion)
+      end
+
+      image_generation
+    end
+
+    def score(trace_id:, name:, value:, id: "#{trace_id}-#{name}")
+      raise ObservabilityUnavailableError, "Observability project slug not configured" unless observability_available?
+
+      if @langfuse_client.nil?
+        raise LangfuseClientNotFoundError,
+              "No Langfuse client found for slug: #{observability_project_slug}"
+      end
+
+      @langfuse_client.create_score(
+        id:,
+        trace_id:,
+        name:,
+        value:,
+        environment: NitroIntelligence.environment
+      )
+    end
+
+    def create_dataset_item(attributes)
+      HTTParty.post("#{@observability_host}/api/public/dataset-items",
+                    body: attributes.to_json,
+                    headers: {
+                      "Content-Type" => "application/json",
+                      "Authorization" => "Basic #{observability_auth_token}",
+                    })
+    end
+
+  private
+
+    attr_reader :observability_project_slug
+
+    def current_revision
+      return @current_revision if defined?(@current_revision)
+
+      path = Rails.root.join("REVISION")
+      @current_revision = File.exist?(path) ? File.read(path).strip.presence : nil
+    end
+
+    def observability_available?
+      observability_project_slug.present?
+    end
+
+    def method_missing(method_name, *, &)
+      @client.send(method_name, *, &)
+    end
+
+    def respond_to_missing?(method_name, include_private = false)
+      @client.respond_to?(method_name, include_private) || super
+    end
+
+    def project_config
+      return @project_config if @project_config.present?
+
+      projects = NitroIntelligence.config.observability_projects
+      @project_config = projects.find { |project| project["slug"] == observability_project_slug }
+
+      if @project_config.nil?
+        raise ObservabilityProjectConfigNotFoundError,
+              "No observability project config found for slug: #{observability_project_slug}"
+      end
+
+      @project_config
+    end
+
+    def client_chat(parameters:)
+      # When requesting an OpenAI model, OpenAI API will return a 400 because it does not ignore custom params
+      @client.chat.completions.create(**parameters.except(*NitroIntelligence.omit_params))
+    end
+
+    def chat_with_tracing(parameters:)
+      project = get_project(
+        project_id: project_config["id"],
+        observability_public_key: project_config["public_key"]
+      )
+      prompt = handle_prompt(parameters:, project_config:)
+
+      instrument_tracing(prompt:, project:, parameters:)
+    rescue ObservabilityProjectNotFoundError, LangfuseClientNotFoundError => e
+      # We should still send the request if we have problems with observability
+      NitroIntelligence.logger.warn(
+        "#{self.class} - Observability configuration provided, but could not be processed. #{e}. " \
+        "Sending request regardless."
+      )
+      client_chat(parameters:)
+    end
+
+    def handle_prompt(parameters:, project_config:)
+      return if parameters[:prompt_name].blank?
+
+      prompt_store = NitroIntelligence::PromptStore.new(
+        observability_project_slug:,
+        observability_public_key: project_config["public_key"],
+        observability_secret_key: project_config["secret_key"]
+      )
+      prompt = prompt_store.get_prompt(
+        prompt_name: parameters[:prompt_name],
+        prompt_label: parameters[:prompt_label],
+        prompt_version: parameters[:prompt_version]
+      )
+      prompt_variables = parameters[:prompt_variables] || {}
+
+      if prompt.present?
+        parameters[:messages] = prompt.interpolate(
+          messages: parameters[:messages],
+          variables: prompt_variables
+        )
+
+        parameters.merge!(prompt.config) unless parameters[:prompt_config_disabled]
+      end
+
+      prompt
+    end
+
+    def instrument_tracing(prompt:, project:, parameters:) # rubocop:disable Metrics/AbcSize, Metrics/MethodLength
+      if @langfuse_client.nil?
+        raise LangfuseClientNotFoundError,
+              "No Langfuse client found for slug: #{observability_project_slug}"
+      end
+
+      default_trace_name = project["name"]
+      input = parameters[:messages]
+      image_generation = parameters[:image_generation]
+      metadata = parameters[:metadata]
+
+      if prompt
+        metadata[:prompt_name] = prompt.name
+        metadata[:prompt_version] = prompt.version
+        default_trace_name = prompt.name
+      end
+
+      seed = parameters[:trace_seed]
+      trace_id = NitroIntelligence::Trace.create_id(seed:) if seed.present?
+
+      chat_completion = nil
+
+      Langfuse.propagate_attributes(
+        user_id: parameters[:user_id] || "default-user",
+        metadata: metadata.transform_values(&:to_s)
+      ) do
+        @langfuse_client.observe(
+          "llm-response",
+          as_type: :generation,
+          trace_id:,
+          environment: NitroIntelligence.environment.to_s,
+          input:,
+          model: parameters[:model],
+          metadata: metadata.transform_values(&:to_s)
+        ) do |generation|
+          generation.update_trace(
+            name: parameters[:trace_name] || default_trace_name,
+            release: current_revision
+          )
+
+          if prompt
+            generation.update({
+                                prompt: {
+                                  name: prompt.name,
+                                  version: prompt.version,
+                                },
+                              })
+          end
+
+          chat_completion = client_chat(parameters:)
+          output = chat_completion.choices.first.message.to_h
+
+          # Handle image generation media
+          if image_generation
+            image_generation.trace_id = generation.trace_id
+            image_generation.parse_file(chat_completion)
+            handle_image_generation_uploads(input, output, image_generation)
+          end
+
+          # Handle truncating any unnecessary data before storing into trace
+          handle_truncation(input, output, chat_completion.model)
+
+          generation.model = chat_completion.model
+          generation.usage_details = {
+            prompt_tokens: chat_completion.usage.prompt_tokens,
+            completion_tokens: chat_completion.usage.completion_tokens,
+            total_tokens: chat_completion.usage.total_tokens,
+          }
+          generation.input = input
+          generation.output = output
+
+          generation.update_trace(input:, output:)
+        end
+      end
+
+      chat_completion
+    end
+
+    # Returns name and metadata
+    def get_project(project_id:, observability_public_key:)
+      cache_key = OBSERVABILITY_PROJECTS_CACHE_KEY_PREFIX + project_id
+      cached_project = NitroIntelligence.cache.read(cache_key)
+      return cached_project if cached_project.present?
+
+      target_project = get_projects(observability_public_key:).find do |project|
+        project["id"] == project_id
+      end
+
+      raise ObservabilityProjectNotFoundError, "Project with ID: #{project_id} not found" if target_project.nil?
+
+      NitroIntelligence.cache.write(cache_key, target_project, expires_in: 12.hours)
+      target_project
+    end
+
+    def get_projects(observability_public_key:)
+      response = HTTParty.get("#{@observability_host}/api/public/projects",
+                              headers: {
+                                "Authorization" => "Basic #{observability_auth_token}",
+                              })
+      data = JSON.parse(response.body)["data"]
+      if data.nil?
+        raise(
+          ObservabilityProjectNotFoundError,
+          "No projects were found. Public key: #{observability_public_key || 'missing'}"
+        )
+      end
+      data
+    end
+
+    def handle_image_generation_uploads(input, output, image_generation)
+      # If we are doing image generation we should upload the media to observability manually
+      upload_handler = NitroIntelligence::UploadHandler.new(@observability_host, observability_auth_token)
+      upload_handler.upload(
+        image_generation.trace_id,
+        upload_queue: Queue.new(image_generation.files)
+      )
+
+      # Replace base64 strings with media references
+      upload_handler.replace_base64_with_media_references(input)
+      upload_handler.replace_base64_with_media_references(output)
+    end
+
+    def handle_truncation(_input, output, model_name)
+      model = NitroIntelligence.model_catalog.lookup_by_name(model_name)
+
+      return unless model&.omit_output_fields
+
+      model.omit_output_fields.each do |omit_output_field|
+        last_key = omit_output_field.last
+        parent_keys = omit_output_field[0...-1]
+        parent = parent_keys.empty? ? output : output.dig(*parent_keys)
+
+        parent[last_key] = "[Truncated...]" if parent.is_a?(Hash) && parent.key?(last_key)
+      end
+    end
+
+    def observability_auth_token
+      public_key = project_config["public_key"]
+      secret_key = project_config["secret_key"]
+      @observability_auth_token ||= Base64.strict_encode64("#{public_key}:#{secret_key}")
+    end
+  end
+end

data/lib/nitro_intelligence/configuration.rb

@@ -0,0 +1,25 @@
+require "logger"
+
+require "nitro_intelligence/null_cache"
+
+module NitroIntelligence
+  class Configuration
+    include ActiveSupport::Configurable
+
+    config_accessor :logger, default: Logger.new($stdout)
+    config_accessor :cache_provider, default: NitroIntelligence::NullCache.new
+    config_accessor :environment, default: "test"
+    config_accessor :agent_server_config, default: {}
+    config_accessor :inference_api_key, default: ""
+    config_accessor :inference_base_url, default: ""
+    config_accessor :model_config, default: {}
+    config_accessor :observability_base_url, default: ""
+    config_accessor :observability_projects, default: []
+
+    class << self
+      def configure
+        yield config
+      end
+    end
+  end
+end