nitro_intelligence 4.2.0 → 4.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 4f03c055275bc0a35eda96d094085e42cefcedf07a9b02e621ecf1ba88bb32f7
4
- data.tar.gz: f11cf2e9e924eff3fd859f9d43a99eaf9993aa11f83308df4354ce717db32627
3
+ metadata.gz: 27f08824317caeb5e6b8de73db8cec8bbd9633a6dca72fc96d37977f7d4776e6
4
+ data.tar.gz: 2d751c4d40c30148d47bf75ac36b0b4a7024b7740b62985f79d2b2d581efd4c0
5
5
  SHA512:
6
- metadata.gz: bbd1c842b1ff8fadc8d86d2090cd8945ebf71cb7354bb815866d0082a383bb4c07113fd3323db420f28db9f7761b2e588cac2c7dd9fd4f294357c86bcc40b53e
7
- data.tar.gz: 8f0a724e2c3d44c0370e333e9dd65842dd4a318c47aa5039feb7c8070b062410ec91337fb305b7fb5133afbafac510f4e6293c59e7c2b47dc9a305357eeacd32
6
+ metadata.gz: aae6508c9d5f23520d46acf912f9dcb4f18643e7950052a24c99f91acb0f99f8ef84156eea767733f1e50314b758dd10d982d8acab10de9c85e1cba0a0a4efa3
7
+ data.tar.gz: 637cb07fea164e376e4c6ca24ad1642bb98e9577c6cc6ea60429c1af5d58ca1f389a8113e71143b7eb96a292d54325758daa4a67af7bbe9f741295ffce233bb8
data/docs/CHANGELOG.md CHANGED
@@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [4.3.0](https://github.com/powerhome/nitro-intelligence.rb/compare/v4.2.0...v4.3.0) (2026-11-11)
9
+
10
+
11
+ ### Added
12
+
13
+ * embed queries and documents with tracing and cost ([#199](https://github.com/powerhome/nitro-intelligence.rb/issues/199)) ([8136c50](https://github.com/powerhome/nitro-intelligence.rb/commit/8136c5076c0f39698bb2d93dd535aad6efbd343e))
14
+ * Expose Cerebro prompts ([#201](https://github.com/powerhome/nitro-intelligence.rb/issues/201)) ([b56cea8](https://github.com/powerhome/nitro-intelligence.rb/commit/b56cea839c7b32aeb6c490ff0c3316f5e7117b21))
15
+ * rerank documents against a query with tracing and cost ([#202](https://github.com/powerhome/nitro-intelligence.rb/issues/202)) ([4f59f4d](https://github.com/powerhome/nitro-intelligence.rb/commit/4f59f4d3458694e3ff45338a2decb973c6015e60))
16
+
8
17
  ## [4.2.0](https://github.com/powerhome/nitro-intelligence.rb/compare/v4.1.1...v4.2.0) (2026-11-10)
9
18
 
10
19
 
data/docs/README.md CHANGED
@@ -54,6 +54,8 @@ NitroIntelligence.configure do |config|
54
54
  "default_image_model" => "google/gemini-3.1-flash-image",
55
55
  "default_text_to_speech_model" => "openai/gpt-4o-mini-tts",
56
56
  "default_decision_model" => "typesafe/jev-1.14",
57
+ "default_embedding_model" => "Qwen/Qwen3-Embedding-0.6B",
58
+ "default_rerank_model" => "Qwen/Qwen3-Reranker-0.6B",
57
59
  "models" => [
58
60
  {
59
61
  "name" => "Qwen/Qwen3.6-35B-A3B",
@@ -81,6 +83,15 @@ NitroIntelligence.configure do |config|
81
83
  {
82
84
  "name" => "typesafe/jev-1.14",
83
85
  "type" => "decision"
86
+ },
87
+ {
88
+ "name" => "Qwen/Qwen3-Embedding-0.6B",
89
+ "type" => "embedding",
90
+ "query_prefix" => "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:"
91
+ },
92
+ {
93
+ "name" => "Qwen/Qwen3-Reranker-0.6B",
94
+ "type" => "rerank"
84
95
  }
85
96
  ]
86
97
  }
@@ -103,7 +114,7 @@ end
103
114
  | `observability_projects` | `Array<Hash>` | `[]` | Langfuse project credentials (slug, id, public_key, secret_key). Add `"mask_content" => true` to keep a project's prompts and responses out of Cerebro and the gateway logs, see [Masking Content](#masking-content) |
104
115
  | `observability_user_id` | `String` | `""` | User ID recorded on observed traces when a request passes no `user_id` parameter — see [Users](#users) |
105
116
  | `assistants_config` | `Hash` | `{}` | Assistants to make addressable by key. `base_url` (String, default: `"https://assistants.powerhome.ai"`) and `user_id` (String, default: `"default-user"`) are shared by every entry; `definitions` (Hash) holds one entry per assistant, keyed by what it is looked up with, each able to override a shared value. Without `definitions` the hash is read as credentials for a single `Assistants.new`, which takes the same two defaults — see [Assistants](#assistants) |
106
- | `model_config` | `Hash` | `{}` | Model defaults and per-model settings. Top-level keys: `default_text_model`, `default_audio_transcription_model`, `default_image_model`, `default_text_to_speech_model`, `default_decision_model`, and `models` (array of per-model hashes keyed by `name` and `type`, with type-specific options like `aspect_ratios`/`resolutions` for images or `voices`/`response_formats` for TTS) |
117
+ | `model_config` | `Hash` | `{}` | Model defaults and per-model settings. Top-level keys: `default_text_model`, `default_audio_transcription_model`, `default_image_model`, `default_text_to_speech_model`, `default_decision_model`, `default_embedding_model`, `default_rerank_model`, and `models` (array of per-model hashes keyed by `name` and `type`, with type-specific options like `aspect_ratios`/`resolutions` for images or `voices`/`response_formats` for TTS, and for embeddings the optional, model-specific `query_prefix` and `document_prefix`: Qwen3-Embedding takes only `query_prefix`, E5 both, OpenAI's models neither, as [Embeddings](#embeddings) describes) |
107
118
 
108
119
  ## Basic Usage
109
120
 
@@ -262,6 +273,55 @@ The model defaults to `default_decision_model` and must be listed in `config.mod
262
273
 
263
274
  The client does not limit the request. To keep a feature portable to models NIP hosts itself, stay within the gateway's [portable subset](https://github.com/powerhome/nitro-intelligence/blob/main/docs/inference-gateway.md#stay-within-what-every-backend-supports): up to 64 questions, 26 options per `choice` question and 10 levels per `score` question, with text `state`.
264
275
 
276
+ ### Embeddings
277
+
278
+ An embeddings model turns text into a vector, so text with similar meaning can be found by comparing vectors. Embeddings go through the gateway like any other request. The gateway must serve an embeddings model; see the [Inference Gateway docs](https://github.com/powerhome/nitro-intelligence/blob/main/docs/inference-gateway.md) for what it serves.
279
+
280
+ Many embeddings models are trained to embed a search query differently from the documents it is searched against, so the client has a method for each. `embed_documents` embeds the documents for an index, and `embed_query` the query to search it with. Both return a `NitroIntelligence::Embeddings` with one vector per input, in input order:
281
+
282
+ ```ruby
283
+ # Documents, such as an index rebuild
284
+ embeddings = client.embed_documents(documents: pages.map(&:description), parameters: { dimensions: 512 })
285
+ embeddings.vectors # => [[0.012, -0.034, ...], ...], one per page
286
+
287
+ # A query to search them with
288
+ client.embed_query(query: "How do I request time off?", parameters: { dimensions: 512 }).vector # => [0.021, 0.008, ...]
289
+ ```
290
+
291
+ `embed_query` also takes a list of queries, such as an evaluation set, and returns a vector for each in `vectors`. `embed_documents` also takes a single document.
292
+
293
+ `parameters` takes `model`, `dimensions`, `batch_size`, `user`, `metadata`, `request_options` and, with observability, `trace_name`, `trace_seed`, `user_id`, `session_id` and `tags`. The model defaults to `default_embedding_model` and must be listed in `config.model_config` with the type `embedding`.
294
+
295
+ * **`dimensions`** shortens the vectors, for a model that supports it, as Qwen3-Embedding does. Embed queries and documents with the same model and the same `dimensions`, or the vectors cannot be compared.
296
+ * **`batch_size`** caps how many inputs go in one request, 128 by default. A longer list is sent in several requests and returned as one result, recorded as one observation, so an index rebuild of a few thousand entries is one call.
297
+
298
+ A model's `query_prefix` and `document_prefix` in `config.model_config` are what it expects before a query's text and before a document's, in the form it was trained on. Qwen3-Embedding takes an instruction before each query and nothing before a document, as in the configuration above; E5 takes `"query: "` and `"passage: "`. Leave both out for a model that embeds queries and documents the same way, such as OpenAI's. Switching to another embeddings model only means changing its `model_config` entry, but vectors from different models cannot be compared, so it also means embedding your stored documents again.
299
+
300
+ `embeddings.usage` holds `prompt_tokens` and `total_tokens` summed over every request, and `embeddings.responses` the gateway's responses, one per batch, each with its `last_response`. With observability enabled, each call is recorded in Cerebro as an embedding observation named `embed-query` or `embed-documents`, with the input as the model received it, prefix included, the gateway's cost added up over every batch, and the number and size of the vectors as output. The vectors themselves are never recorded. If the gateway did not price every batch, the cost is left unset rather than undercounted. If a batch fails after others succeeded, the failed call still records what those cost before the error is raised.
301
+
302
+ Embeddings take no managed prompt: `prompt_name` and `prompt_variables` are ignored.
303
+
304
+ ### Reranking
305
+
306
+ A reranker scores how relevant each of a list of documents is to a query. It is more accurate than comparing vectors, and suits reordering the few dozen candidates a vector search returned. The gateway must serve a reranker; see the [Inference Gateway docs](https://github.com/powerhome/nitro-intelligence/blob/main/docs/inference-gateway.md) for what it serves.
307
+
308
+ `rerank` takes a query and a list of documents and returns a `NitroIntelligence::Ranking`, most relevant first:
309
+
310
+ ```ruby
311
+ ranking = client.rerank(
312
+ query: "How do I request time off?",
313
+ documents: candidates.map(&:description),
314
+ parameters: { top_n: 5 }
315
+ )
316
+ ranking.results.first.index # => 3, the position in `documents`
317
+ ranking.results.first.relevance_score # => 0.97, from 0 to 1
318
+ candidates.values_at(*ranking.results.map(&:index))
319
+ ```
320
+
321
+ `parameters` takes `model`, `top_n` (return only the most relevant this many, a positive integer, applied after sorting even when the provider ignores it), `metadata`, `request_options` and, with observability, `trace_name`, `trace_seed`, `user_id`, `session_id`, `tags` and `trace_documents`. The model defaults to `default_rerank_model` and must be listed in `config.model_config` with the type `rerank`. `ranking.meta` holds what the provider billed, and `ranking.last_response` the HTTP response, including the gateway's headers.
322
+
323
+ With observability enabled, each call is recorded in Cerebro as a generation, since Langfuse has no reranker type, with the query and documents as input, each document's index and score as output, the input tokens as usage, any search units the provider billed in the metadata, and the gateway's cost. Pass `trace_documents: false` when your own trace already records the candidates, such as an embedding call or a search step in the same trace, and only their number is recorded. Reranking takes no managed prompt: `prompt_name` and `prompt_variables` are ignored.
324
+
265
325
  ### Image Editing and Generation
266
326
 
267
327
  Nitro Intelligence can be used for image editing and generation
@@ -526,7 +586,7 @@ success, so it records no input up front.
526
586
 
527
587
  A project can opt in to recording what happened without recording what was said. Set `"mask_content" => true` on its entry in `observability_projects`; projects that do not set it are unchanged.
528
588
 
529
- For a masked project, every observed request (chat, image generation, audio transcription, text-to-speech and decisions) is still traced, with its model, usage, cost, prompt link, tags, user, session, metadata and gateway call ID, but:
589
+ For a masked project, every observed request (chat, image generation, audio transcription, text-to-speech, decisions, embeddings and reranking) is still traced, with its model, usage, cost, prompt link, tags, user, session, metadata and gateway call ID, but:
530
590
 
531
591
  * The input and output of the generation and of the trace are recorded as `[masked]`. The whole value is replaced, so reasoning, tool calls and anything else in a response are masked however the provider shaped it.
532
592
  * A failed request records the error class (for example `OpenAI::Errors::BadRequestError`) as its status message, without the error's own message, which can quote the request or the response.
@@ -648,6 +708,8 @@ trace_url = "#{NitroIntelligence.config.observability_base_url}/project/#{projec
648
708
  request["Authorization"] = "Basic #{project.auth_token}"
649
709
  ```
650
710
 
711
+ `project.prompt(name, label:, version:)` fetches one of the project's prompts, as [`client.prompt`](#reading-a-prompt) does.
712
+
651
713
  `NitroIntelligence.project_client_registry.fetch(slug)` builds each project's client once per process, and raises `NitroIntelligence::Observability::Project::NotFoundError` for a slug `observability_projects` does not configure. `NitroIntelligence::Observability::Project.find_by_slug(slug:)` reads the same configuration without building a client, and returns `nil` instead. The gem never raises `NitroIntelligence::Observability::ProjectClient::NotFoundError`; it is public only because applications raise it themselves, and is to be retired ([#182](https://github.com/powerhome/nitro-intelligence.rb/issues/182)). Rescue `Project::NotFoundError` for a project that is not configured.
652
714
 
653
715
  Traces and scores are sent to Cerebro in the background. Fetch each project in an initializer to build its client at boot, and shut them all down as the process exits so that the last ones are sent:
@@ -788,6 +850,25 @@ client.chat(
788
850
  )
789
851
  ```
790
852
 
853
+ #### Reading a Prompt
854
+
855
+ To read a prompt without making a model call, for its config or to compile its text for a request of your own, fetch it from an observed client:
856
+
857
+ ```ruby
858
+ client = NitroIntelligence::Client.new(observability_project_slug: "page-finder")
859
+ prompt = client.prompt("Page Finder Search") # the production label
860
+ prompt = client.prompt("Page Finder Search", label: "staging")
861
+ prompt = client.prompt("Page Finder Search", version: 4)
862
+
863
+ prompt.name # => "Page Finder Search"
864
+ prompt.version # => 4
865
+ prompt.type # => "text", or "chat"
866
+ prompt.config # => { min_relevance: 0.1, candidates: 50 }, keys symbolized
867
+ prompt.compile(product: "Nitro") # the text, or the messages, with its variables filled in
868
+ ```
869
+
870
+ Code that holds a project rather than a client can call `project.prompt` with the same arguments (see [Reaching Cerebro Directly](#reaching-cerebro-directly)). Either uses the same cache as a request naming the prompt: five minutes for a label, indefinitely for a version, and the last prompt fetched for a label when Cerebro cannot be reached. To use one prompt's settings across several steps, fetch it once and keep the result, or pass its `version` to later lookups, so that a label moved in between cannot mix two versions. A missing prompt raises `ObservabilityPromptNotFoundError` and a failed lookup `ObservabilityPromptError`, as below. A client without observability has no project to fetch from, and raises `NitroIntelligence::Observability::Project::NotFoundError`.
871
+
791
872
  #### Prompt Errors
792
873
 
793
874
  A request naming a prompt raises when it cannot use one:
@@ -1,8 +1,11 @@
1
1
  require "nitro_intelligence/client/handlers/audio_transcription_handler"
2
2
  require "nitro_intelligence/client/handlers/chat_handler"
3
3
  require "nitro_intelligence/client/handlers/decision_handler"
4
+ require "nitro_intelligence/client/handlers/embedding_handler"
4
5
  require "nitro_intelligence/client/handlers/image_handler"
6
+ require "nitro_intelligence/client/handlers/rerank_handler"
5
7
  require "nitro_intelligence/client/handlers/text_to_speech_handler"
8
+ require "nitro_intelligence/observability/project"
6
9
 
7
10
  module NitroIntelligence
8
11
  module Client
@@ -60,6 +63,81 @@ module NitroIntelligence
60
63
  decision_handler.create(state:, questions:, parameters: parameters.dup)
61
64
  end
62
65
 
66
+ # Embeds a search query as a vector over the gateway's `/v1/embeddings` route, to compare
67
+ # with vectors from {#embed_documents}.
68
+ #
69
+ # @param query [String, Array<String>] the query, or several, such as an evaluation set
70
+ # @param parameters [Hash] `model`, `dimensions` (a shorter vector, for a model that supports
71
+ # one), `batch_size` (at most this many texts per request, 128 by default), `user`,
72
+ # `metadata` and `request_options`, and on an observed client `trace_name`, `trace_seed`,
73
+ # `user_id`, `session_id` and `tags`. It takes no managed prompt.
74
+ # @return [Embeddings] the query's vector in `vector`, or one per query in `vectors`
75
+ # @raise [ArgumentError] when the model is not one `model_config` lists with the type
76
+ # `embedding`, a query is empty or not text, or `batch_size` is not a positive integer
77
+ # @example
78
+ # client.embed_query(query: "How do I request time off?").vector
79
+ def embed_query(query:, parameters: {})
80
+ embedding_handler.create(input: query, input_type: :query, parameters: parameters.dup)
81
+ end
82
+
83
+ # Embeds documents as vectors over the gateway's `/v1/embeddings` route, for a search index.
84
+ #
85
+ # @param documents [Array<String>, String] the documents. A long list is sent in batches of
86
+ # `batch_size` and recorded as one observation.
87
+ # @param parameters [Hash] the same as {#embed_query}'s
88
+ # @return [Embeddings] one vector per document in `vectors`, in the order given
89
+ # @raise [ArgumentError] as {#embed_query} does
90
+ # @example
91
+ # client.embed_documents(documents: pages.map(&:description), parameters: { dimensions: 512 }).vectors
92
+ def embed_documents(documents:, parameters: {})
93
+ embedding_handler.create(input: documents, input_type: :document, parameters: parameters.dup)
94
+ end
95
+
96
+ # Ranks documents by their relevance to a query over the gateway's rerank route,
97
+ # `POST /v1/rerank`.
98
+ #
99
+ # @param query [String] what the documents are ranked against
100
+ # @param documents [Array<String>] the candidates
101
+ # @param parameters [Hash] `model`, `top_n` (return only the most relevant this many),
102
+ # `metadata` and `request_options`, and on an observed client `trace_name`, `trace_seed`,
103
+ # `user_id`, `session_id`, `tags` and `trace_documents` (false to record only how many
104
+ # documents there were, for a caller whose own trace already records them). It takes no
105
+ # managed prompt.
106
+ # @return [Ranking]
107
+ # @raise [ArgumentError] when the model is not one `model_config` lists with the type
108
+ # `rerank`, the query is empty, the documents are empty or not text, or `top_n` is not a
109
+ # positive integer
110
+ # @example
111
+ # client.rerank(query: "How do I request time off?", documents: pages.map(&:description),
112
+ # parameters: { top_n: 5 }).results.map(&:index)
113
+ def rerank(query:, documents:, parameters: {})
114
+ rerank_handler.create(query:, documents:, parameters: parameters.dup)
115
+ end
116
+
117
+ # Fetches a prompt from the client's Cerebro project without making a model call, as
118
+ # {Observability::Project#prompt} does.
119
+ #
120
+ # @param name [String] the prompt's name in Cerebro
121
+ # @param label [String, nil] the label to fetch, `"production"` when neither it nor `version`
122
+ # is given
123
+ # @param version [Integer, nil] the version to fetch, taking precedence over `label`
124
+ # @return [Observability::Prompt]
125
+ # @raise [Observability::Project::NotFoundError] when the client has no Cerebro project: it was
126
+ # built without an `observability_project_slug`, or with one `observability_projects` does
127
+ # not configure
128
+ # @raise [Observability::PromptStore::ObservabilityPromptNotFoundError] when the prompt, at that
129
+ # label or version, does not exist
130
+ # @raise [Observability::PromptStore::ObservabilityPromptError] when Cerebro fails to return
131
+ # the prompt
132
+ # @example
133
+ # client = NitroIntelligence::Client.new(observability_project_slug: "page-finder")
134
+ # client.prompt("Page Finder Search").config[:min_relevance] # => 0.1
135
+ def prompt(name, label: nil, version: nil) # rubocop:disable Lint/UnusedMethodArgument
136
+ raise Observability::Project::NotFoundError,
137
+ "This client has no Cerebro project to fetch #{name.inspect} from: build it with an " \
138
+ "observability_project_slug that observability_projects configures"
139
+ end
140
+
63
141
  # Generates an image, or edits `target_image`.
64
142
  #
65
143
  # @param message [String] what to generate, or how to edit the target
@@ -122,6 +200,10 @@ module NitroIntelligence
122
200
  @decision_handler ||= Handlers::DecisionHandler.new(client: @client)
123
201
  end
124
202
 
203
+ def embedding_handler
204
+ @embedding_handler ||= Handlers::EmbeddingHandler.new(client: @client)
205
+ end
206
+
125
207
  def image_handler
126
208
  @image_handler ||= Handlers::ImageHandler.new(client: @client)
127
209
  end
@@ -134,6 +216,10 @@ module NitroIntelligence
134
216
  @client.respond_to?(method_name, include_private) || super
135
217
  end
136
218
 
219
+ def rerank_handler
220
+ @rerank_handler ||= Handlers::RerankHandler.new(client: @client)
221
+ end
222
+
137
223
  def text_to_speech_handler
138
224
  @text_to_speech_handler ||= Handlers::TextToSpeechHandler.new(client: @client)
139
225
  end
@@ -1,8 +1,8 @@
1
1
  require "nitro_intelligence/client/factory"
2
2
 
3
3
  module NitroIntelligence
4
- # Inference through the inference gateway: chat, image generation and editing, audio
5
- # transcription and text-to-speech. See the README's "Basic Usage".
4
+ # Inference through the inference gateway: chat, decisions, embeddings, reranking, image
5
+ # generation and editing, audio transcription and text-to-speech. See the README's "Basic Usage".
6
6
  module Client
7
7
  # Builds a client, which records every request in Cerebro when given a project to report to.
8
8
  #
@@ -0,0 +1,95 @@
1
+ require "openai"
2
+ require "nitro_intelligence/client/handlers/base_handler"
3
+ require "nitro_intelligence/embeddings"
4
+
5
+ module NitroIntelligence
6
+ module Client
7
+ module Handlers
8
+ # Embeds text over the gateway's `/v1/embeddings` route, splitting a long list of inputs into
9
+ # batches of `batch_size` so an index rebuild of thousands of entries is one call.
10
+ #
11
+ # @api private
12
+ class EmbeddingHandler < BaseHandler
13
+ # Fields of openai-ruby's embedding request taken from `parameters`. The input is the
14
+ # handler's own argument, and the vectors are always requested as floats, the only
15
+ # encoding openai-ruby decodes into the response it returns.
16
+ ALLOWED_EXTRA_PARAMETERS =
17
+ (OpenAI::Models::EmbeddingCreateParams.fields.keys - %i[input encoding_format]).uniq.freeze
18
+
19
+ DEFAULT_BATCH_SIZE = 128
20
+
21
+ INPUT_TYPES = %i[query document].freeze
22
+
23
+ def create(input:, input_type:, parameters: {})
24
+ validate_and_resolve!(input, input_type, parameters)
25
+ perform_request(input:, input_type:, parameters:)
26
+ end
27
+
28
+ # Each batch's response is added to `responses` as it arrives, so a caller can still
29
+ # account for the batches already billed when a later one raises.
30
+ def perform_request(input:, input_type:, parameters: {}, correlation_trace_id: nil, mask_content: false,
31
+ responses: [])
32
+ add_gateway_headers(parameters, correlation_trace_id:, mask_content:)
33
+ texts(input, input_type, parameters).each_slice(parameters[:batch_size]) do |batch|
34
+ responses << @client.embeddings.create(
35
+ input: batch, encoding_format: :float, **parameters.slice(*ALLOWED_EXTRA_PARAMETERS)
36
+ )
37
+ end
38
+ Embeddings.new(responses)
39
+ end
40
+
41
+ def validate_and_resolve!(input, input_type, parameters)
42
+ default_parameters = {
43
+ metadata: {},
44
+ model: NitroIntelligence.model_catalog.default_embedding_model&.name,
45
+ batch_size: DEFAULT_BATCH_SIZE,
46
+ }
47
+
48
+ parameters.replace(default_parameters.merge(parameters))
49
+ Client.validate_model(parameters[:model], type: EmbeddingModel)
50
+ unless INPUT_TYPES.include?(input_type)
51
+ raise ArgumentError, "input_type must be one of #{INPUT_TYPES.inspect}"
52
+ end
53
+
54
+ validate_input!(input)
55
+ validate_batch_size!(parameters[:batch_size])
56
+ end
57
+
58
+ # The texts as the model receives them: each after the model's prefix for queries or for
59
+ # documents, which is empty for a model that embeds both the same way.
60
+ def texts(input, input_type, parameters)
61
+ prefix = Client.validate_model(parameters[:model], type: EmbeddingModel).prefix_for(input_type)
62
+ Array(input).map { |text| "#{prefix}#{text}" }
63
+ end
64
+
65
+ # The cost of every batch, added up. A batch the gateway did not price leaves the whole call
66
+ # unpriced, because a sum that silently skips it would understate the spend.
67
+ def cost_details(embeddings)
68
+ costs = embeddings.responses.map { |response| super(response) }
69
+ return nil if costs.empty? || costs.any?(&:nil?)
70
+
71
+ %i[total input output].index_with do |component|
72
+ costs.sum { |cost| cost[component] } if costs.all? { |cost| cost.key?(component) }
73
+ end.compact
74
+ end
75
+
76
+ private
77
+
78
+ # Checked before any batch is sent, so a bad entry late in a long list cannot fail the call
79
+ # after the batches before it were billed.
80
+ def validate_input!(input)
81
+ texts = Array(input)
82
+ return if texts.any? && texts.all? { |text| text.is_a?(String) && text.present? }
83
+
84
+ raise ArgumentError, "input must be a non-empty string or a non-empty array of non-empty strings"
85
+ end
86
+
87
+ def validate_batch_size!(batch_size)
88
+ return if batch_size.is_a?(Integer) && batch_size.positive?
89
+
90
+ raise ArgumentError, "batch_size must be a positive integer, not #{batch_size.inspect}"
91
+ end
92
+ end
93
+ end
94
+ end
95
+ end
@@ -0,0 +1,73 @@
1
+ require "nitro_intelligence/client/handlers/observed/base_handler"
2
+
3
+ module NitroIntelligence
4
+ module Client
5
+ module Handlers
6
+ module Observed
7
+ # Recorded as one embedding observation per call, however many batches the input was sent
8
+ # in, so an index rebuild is one observation rather than one for each entry.
9
+ #
10
+ # @api private
11
+ class EmbeddingHandler < BaseHandler
12
+ OBSERVATION_NAMES = { query: "embed-query", document: "embed-documents" }.freeze
13
+
14
+ def create(input:, input_type:, parameters: {})
15
+ @base_handler.validate_and_resolve!(input, input_type, parameters)
16
+
17
+ trace_name = parameters[:trace_name] || project.slug
18
+ texts = @base_handler.texts(input, input_type, parameters)
19
+
20
+ # The texts as the model received them, prefix included, since the prefix changes the
21
+ # vector as much as the text does.
22
+ @observer.observe(
23
+ OBSERVATION_NAMES.fetch(input_type),
24
+ type: :embedding,
25
+ parameters:,
26
+ trace_name:,
27
+ input: input.is_a?(Array) ? texts : texts.first
28
+ ) do |embedding|
29
+ workflow(embedding:, input:, input_type:, parameters:)
30
+ end
31
+ end
32
+
33
+ private
34
+
35
+ def workflow(embedding:, input:, input_type:, parameters:)
36
+ responses = []
37
+ embeddings = @base_handler.perform_request(
38
+ input:, input_type:, parameters:,
39
+ correlation_trace_id: embedding.trace_id, mask_content: project.mask_content?, responses:
40
+ )
41
+
42
+ # The vectors are never recorded: they are large, mean nothing to read, and are the
43
+ # caller's to store. Their count and size say whether the call did what was asked.
44
+ trace_attributes = {
45
+ model: embeddings.model,
46
+ output: { count: embeddings.vectors.size, dimensions: embeddings.vector&.size },
47
+ usage_details: embeddings.usage.to_h.symbolize_keys.presence,
48
+ cost_details: @base_handler.cost_details(embeddings),
49
+ }
50
+
51
+ [embeddings, trace_attributes]
52
+ rescue
53
+ record_billed_batches(embedding, responses)
54
+ raise
55
+ end
56
+
57
+ # A batch that fails after others succeeded leaves those billed, so the failed call still
58
+ # records what they used and cost, rather than nothing, before the error is raised.
59
+ def record_billed_batches(embedding, responses)
60
+ return if responses.empty?
61
+
62
+ billed = Embeddings.new(responses)
63
+ attributes = {
64
+ usage_details: billed.usage.to_h.symbolize_keys.presence,
65
+ cost_details: @base_handler.cost_details(billed),
66
+ }.compact
67
+ embedding.update(attributes) if attributes.any?
68
+ end
69
+ end
70
+ end
71
+ end
72
+ end
73
+ end
@@ -0,0 +1,78 @@
1
+ require "nitro_intelligence/client/handlers/observed/base_handler"
2
+
3
+ module NitroIntelligence
4
+ module Client
5
+ module Handlers
6
+ module Observed
7
+ # Recorded as a generation: Langfuse has no reranker observation type, and a generation
8
+ # carries the model, usage and cost, as it does for decisions.
9
+ #
10
+ # @api private
11
+ class RerankHandler < BaseHandler
12
+ def create(query:, documents:, parameters: {})
13
+ @base_handler.validate_and_resolve!(query, documents, parameters)
14
+
15
+ trace_name = parameters[:trace_name] || project.slug
16
+
17
+ @observer.observe(
18
+ "rerank",
19
+ type: :generation,
20
+ parameters:,
21
+ trace_name:,
22
+ input: input(query, documents, parameters)
23
+ ) do |generation|
24
+ workflow(generation:, query:, documents:, parameters:)
25
+ end
26
+ end
27
+
28
+ private
29
+
30
+ # A caller whose trace already records the candidates, as a retrieval step of its own,
31
+ # can pass `trace_documents: false` to record only how many there were.
32
+ def input(query, documents, parameters)
33
+ if parameters[:trace_documents] == false
34
+ { query:, document_count: documents.size }
35
+ else
36
+ { query:, documents: }
37
+ end
38
+ end
39
+
40
+ def workflow(generation:, query:, documents:, parameters:)
41
+ ranking = @base_handler.perform_request(
42
+ query:, documents:, parameters:, correlation_trace_id: generation.trace_id,
43
+ mask_content: project.mask_content?
44
+ )
45
+
46
+ search_units = count(ranking.meta, :billed_units, :search_units)
47
+ generation.metadata = { search_units: } if search_units
48
+
49
+ trace_attributes = {
50
+ model: parameters[:model],
51
+ output: ranking.results.map(&:to_h),
52
+ usage_details: usage_details(ranking.meta),
53
+ cost_details: @base_handler.cost_details(ranking),
54
+ }
55
+
56
+ [ranking, trace_attributes]
57
+ end
58
+
59
+ # Providers report a rerank's size as tokens, search units, or both. The tokens are recorded
60
+ # under Langfuse's own `input` and `total` keys: OpenAI's names are only recognised alongside
61
+ # `completion_tokens`, which a rerank has none of, and Langfuse would otherwise total every
62
+ # counter it was given, adding input tokens to themselves. Search units are not tokens, so
63
+ # they go in the metadata instead. A count the provider sent in some other shape is left
64
+ # out, rather than failing a request already paid for.
65
+ def usage_details(meta)
66
+ input_tokens = count(meta, :tokens, :input_tokens) || count(meta, :billed_units, :total_tokens)
67
+ { input: input_tokens, total: input_tokens } if input_tokens
68
+ end
69
+
70
+ def count(meta, section, key)
71
+ value = meta[section][key] if meta[section].is_a?(Hash)
72
+ value if value.is_a?(Numeric)
73
+ end
74
+ end
75
+ end
76
+ end
77
+ end
78
+ end
@@ -0,0 +1,82 @@
1
+ require "openai"
2
+ require "nitro_intelligence/client/handlers/base_handler"
3
+ require "nitro_intelligence/ranking"
4
+
5
+ module NitroIntelligence
6
+ module Client
7
+ module Handlers
8
+ # Ranks documents by their relevance to a query over the gateway's Cohere-compatible rerank
9
+ # route, `POST /v1/rerank`. openai-ruby has no resource for it, so the request goes through
10
+ # the client's generic `request`, which keeps its credentials, retries, timeouts and errors.
11
+ #
12
+ # @api private
13
+ class RerankHandler < BaseHandler
14
+ PATH = "v1/rerank".freeze
15
+ # What is left of that route to add to an `inference_base_url` that already ends in `/v1`.
16
+ PATH_AFTER_V1 = "rerank".freeze
17
+
18
+ # Rerank request fields other than the model, query and documents, taken from `parameters`.
19
+ BODY_PARAMETERS = %i[top_n].freeze
20
+
21
+ # Declaring a model, rather than taking the decoded hash, is what gets the response its
22
+ # `last_response`, and with it the gateway's cost headers. `Ranking` reads `results` and
23
+ # `meta` raw through `BaseModel#[]`.
24
+ class Response < OpenAI::BaseModel
25
+ required :id, String, nil?: true
26
+ end
27
+
28
+ def create(query:, documents:, parameters: {})
29
+ validate_and_resolve!(query, documents, parameters)
30
+ perform_request(query:, documents:, parameters:)
31
+ end
32
+
33
+ def perform_request(query:, documents:, parameters: {}, correlation_trace_id: nil, mask_content: false)
34
+ add_gateway_headers(parameters, correlation_trace_id:, mask_content:)
35
+ response = @client.request(
36
+ method: :post,
37
+ path:,
38
+ body: { model: parameters[:model], query:, documents:, **parameters.slice(*BODY_PARAMETERS).compact },
39
+ model: Response,
40
+ options: parameters[:request_options]
41
+ )
42
+ Ranking.new(response, top_n: parameters[:top_n])
43
+ end
44
+
45
+ def validate_and_resolve!(query, documents, parameters)
46
+ default_parameters = {
47
+ metadata: {},
48
+ model: NitroIntelligence.model_catalog.default_rerank_model&.name,
49
+ }
50
+
51
+ parameters.replace(default_parameters.merge(parameters))
52
+ Client.validate_model(parameters[:model], type: RerankModel)
53
+ raise ArgumentError, "query must be a non-empty string" unless query.is_a?(String) && query.present?
54
+
55
+ validate_documents!(documents)
56
+ validate_top_n!(parameters[:top_n])
57
+ end
58
+
59
+ private
60
+
61
+ def validate_documents!(documents)
62
+ texts = documents.is_a?(Array) && documents.any? && documents.all?(String)
63
+ return if texts && documents.all?(&:present?)
64
+
65
+ raise ArgumentError, "documents must be a non-empty array of non-empty strings"
66
+ end
67
+
68
+ def validate_top_n!(top_n)
69
+ return if top_n.nil? || (top_n.is_a?(Integer) && top_n.positive?)
70
+
71
+ raise ArgumentError, "top_n must be a positive integer, not #{top_n.inspect}"
72
+ end
73
+
74
+ # Both reach `/v1/rerank`: the whole route on a base URL without a version, or only the
75
+ # rest of it on one that already ends in `/v1`.
76
+ def path
77
+ @client.base_url.path.chomp("/").end_with?("/v1") ? PATH_AFTER_V1 : PATH
78
+ end
79
+ end
80
+ end
81
+ end
82
+ end
@@ -1,7 +1,9 @@
1
1
  require "nitro_intelligence/client/handlers/observed/audio_transcription_handler"
2
2
  require "nitro_intelligence/client/handlers/observed/chat_handler"
3
3
  require "nitro_intelligence/client/handlers/observed/decision_handler"
4
+ require "nitro_intelligence/client/handlers/observed/embedding_handler"
4
5
  require "nitro_intelligence/client/handlers/observed/image_handler"
6
+ require "nitro_intelligence/client/handlers/observed/rerank_handler"
5
7
  require "nitro_intelligence/client/handlers/observed/text_to_speech_handler"
6
8
  require "nitro_intelligence/observability/prompt_store"
7
9
  require "nitro_intelligence/observability/upload_handler"
@@ -16,6 +18,11 @@ module NitroIntelligence
16
18
  @observer = observer
17
19
  end
18
20
 
21
+ # @api private
22
+ def prompt(name, label: nil, version: nil)
23
+ @observer.project_client.project.prompt(name, label:, version:)
24
+ end
25
+
19
26
  private
20
27
 
21
28
  def chat_handler
@@ -37,12 +44,24 @@ module NitroIntelligence
37
44
  )
38
45
  end
39
46
 
47
+ def embedding_handler
48
+ @embedding_handler ||= Handlers::Observed::EmbeddingHandler.new(
49
+ base_handler: Handlers::EmbeddingHandler.new(client: @client), observer: @observer
50
+ )
51
+ end
52
+
40
53
  def image_handler
41
54
  @image_handler ||= Handlers::Observed::ImageHandler.new(
42
55
  base_handler: Handlers::ImageHandler.new(client: @client), observer: @observer
43
56
  )
44
57
  end
45
58
 
59
+ def rerank_handler
60
+ @rerank_handler ||= Handlers::Observed::RerankHandler.new(
61
+ base_handler: Handlers::RerankHandler.new(client: @client), observer: @observer
62
+ )
63
+ end
64
+
46
65
  def text_to_speech_handler
47
66
  @text_to_speech_handler ||= Handlers::Observed::TextToSpeechHandler.new(
48
67
  base_handler: Handlers::TextToSpeechHandler.new(client: @client), observer: @observer
@@ -90,12 +90,23 @@ module NitroIntelligence
90
90
 
91
91
  RECORDED_ATTRIBUTES.each do |name|
92
92
  value = trace_attributes[name]
93
- generation.public_send(:"#{name}=", value) if value
93
+ record_attribute(generation, name, value) if value
94
94
  end
95
95
 
96
96
  record_status(generation, trace_attributes)
97
97
  end
98
98
 
99
+ # langfuse-rb's embedding observation takes its cost only through #update: unlike a
100
+ # generation, it has no setter for it.
101
+ def record_attribute(observation, name, value)
102
+ setter = :"#{name}="
103
+ if observation.respond_to?(setter)
104
+ observation.public_send(setter, value)
105
+ else
106
+ observation.update(name => value)
107
+ end
108
+ end
109
+
99
110
  def masked(trace_attributes)
100
111
  return trace_attributes unless project.mask_content?
101
112
 
@@ -0,0 +1,39 @@
1
+ module NitroIntelligence
2
+ # What {Client::Base#embed_query} and {Client::Base#embed_documents} return: one vector for each
3
+ # input, in the order the inputs were given, however many requests it took to embed them.
4
+ class Embeddings
5
+ # @return [String, nil] the model that embedded the input, as the gateway named it
6
+ attr_reader :model
7
+ # @return [Array<Array<Float>>] one vector for each input, in input order
8
+ attr_reader :vectors
9
+ # @return [ActiveSupport::HashWithIndifferentAccess] `prompt_tokens` and `total_tokens`,
10
+ # summed over every request, each left out when a response did not report it
11
+ attr_reader :usage
12
+ # @return [Array<OpenAI::Models::CreateEmbeddingResponse>] the gateway's responses, one for each
13
+ # batch of input, each with its `last_response` and the gateway's headers
14
+ attr_reader :responses
15
+
16
+ # @api private
17
+ def initialize(responses)
18
+ @responses = responses
19
+ @model = responses.first&.model
20
+ @vectors = responses.flat_map { |response| response.data.sort_by(&:index).map(&:embedding) }
21
+ @usage = %i[prompt_tokens total_tokens].index_with { |count| total(count) }.compact.with_indifferent_access
22
+ end
23
+
24
+ # The vector of the first input: the only one when the input was a single string.
25
+ #
26
+ # @return [Array<Float>, nil]
27
+ def vector
28
+ vectors.first
29
+ end
30
+
31
+ private
32
+
33
+ # Left out when any response did not report it, rather than undercounting.
34
+ def total(count)
35
+ counts = responses.map { |response| response.usage&.public_send(count) }
36
+ counts.sum if counts.any? && counts.none?(&:nil?)
37
+ end
38
+ end
39
+ end
@@ -28,6 +28,27 @@ module NitroIntelligence
28
28
  # @api private
29
29
  class DecisionModel < Model; end
30
30
 
31
+ # @api private
32
+ class EmbeddingModel < Model
33
+ # What the model expects before a query's text and before a document's, for a model trained
34
+ # to embed the two differently. Empty for a model that embeds both the same way.
35
+ attr_reader :query_prefix, :document_prefix
36
+
37
+ def initialize(name:, omit_output_fields: [], query_prefix: "", document_prefix: "", **)
38
+ super
39
+ @query_prefix = query_prefix.to_s
40
+ @document_prefix = document_prefix.to_s
41
+ end
42
+
43
+ # @param input_type [Symbol] `:query` or `:document`
44
+ def prefix_for(input_type)
45
+ input_type == :query ? query_prefix : document_prefix
46
+ end
47
+ end
48
+
49
+ # @api private
50
+ class RerankModel < Model; end
51
+
31
52
  # @api private
32
53
  class TextModel < Model; end
33
54
 
@@ -6,7 +6,9 @@ module NitroIntelligence
6
6
  attr_reader :models,
7
7
  :default_audio_transcription_model,
8
8
  :default_decision_model,
9
+ :default_embedding_model,
9
10
  :default_image_model,
11
+ :default_rerank_model,
10
12
  :default_text_model,
11
13
  :default_text_to_speech_model
12
14
 
@@ -15,7 +17,9 @@ module NitroIntelligence
15
17
  @models = (model_config[:models] || []).map { |model_metadata| ModelFactory.build(model_metadata) }
16
18
  @default_audio_transcription_model = lookup_by_name(model_config[:default_audio_transcription_model])
17
19
  @default_decision_model = lookup_by_name(model_config[:default_decision_model])
20
+ @default_embedding_model = lookup_by_name(model_config[:default_embedding_model])
18
21
  @default_image_model = lookup_by_name(model_config[:default_image_model])
22
+ @default_rerank_model = lookup_by_name(model_config[:default_rerank_model])
19
23
  @default_text_model = lookup_by_name(model_config[:default_text_model])
20
24
  @default_text_to_speech_model = lookup_by_name(model_config[:default_text_to_speech_model])
21
25
  end
@@ -7,7 +7,9 @@ module NitroIntelligence
7
7
  "text" => TextModel,
8
8
  "audio_transcription" => TextModel,
9
9
  "decision" => DecisionModel,
10
+ "embedding" => EmbeddingModel,
10
11
  "image" => ImageModel,
12
+ "rerank" => RerankModel,
11
13
  "text_to_speech" => TextToSpeechModel,
12
14
  }.freeze
13
15
 
@@ -39,6 +39,32 @@ module NitroIntelligence
39
39
  )
40
40
  end
41
41
 
42
+ # Fetches one of the project's prompts, without making a model call: to read its `config`, or to
43
+ # compile its text for a request of your own.
44
+ #
45
+ # A prompt fetched by label is cached for five minutes, and by version indefinitely. If Cerebro
46
+ # cannot be reached, a label lookup falls back to the last prompt fetched for that label, however
47
+ # old. To use the same prompt for several steps, fetch it once and ask for its `version` in the
48
+ # requests that follow, so that a label moved in between cannot mix two versions.
49
+ #
50
+ # @param name [String] the prompt's name in Cerebro
51
+ # @param label [String, nil] the label to fetch, `"production"` when neither it nor `version`
52
+ # is given
53
+ # @param version [Integer, nil] the version to fetch, taking precedence over `label`
54
+ # @return [Prompt]
55
+ # @raise [PromptStore::ObservabilityPromptNotFoundError] when the prompt, at that label or
56
+ # version, does not exist
57
+ # @raise [PromptStore::ObservabilityPromptError] when Cerebro fails to return the prompt
58
+ # @example
59
+ # prompt = project.prompt("Page Finder Search")
60
+ # prompt.config[:candidates] # => 50
61
+ def prompt(name, label: nil, version: nil)
62
+ prompt = prompt_store.get_prompt(prompt_name: name, prompt_label: label, prompt_version: version)
63
+ raise PromptStore::ObservabilityPromptNotFoundError, "Prompt: #{name} Not Found" if prompt.nil?
64
+
65
+ prompt
66
+ end
67
+
42
68
  # @api private
43
69
  def mask_content?
44
70
  @mask_content
@@ -1,11 +1,32 @@
1
1
  module NitroIntelligence
2
2
  module Observability
3
- # @api private
3
+ # A prompt managed in Cerebro, as {Project#prompt} and {Client::Base#prompt} return it. A request
4
+ # that names a prompt with `prompt_name` uses one of these, but an application can also read one
5
+ # itself, for its text or its `config`, without making a model call.
4
6
  class Prompt
5
- attr_reader :name, :type, :prompt, :version, :config, :labels, :tags
7
+ # @return [String] the prompt's name in Cerebro
8
+ attr_reader :name
9
+ # @return [String] `"text"` for a single string, or `"chat"` for a list of role-tagged messages
10
+ attr_reader :type
11
+ # @return [String, Array<Hash>] the prompt's content as Cerebro stores it, variables
12
+ # uncompiled: a string for a text prompt, or messages with `:role` and `:content` for a chat
13
+ # prompt. {#compile} fills in the variables.
14
+ attr_reader :prompt
15
+ # @return [Integer] the version Cerebro returned, which a later lookup can ask for to get the
16
+ # same prompt again
17
+ attr_reader :version
18
+ # @return [Hash{Symbol => Object}] the prompt's config, keys symbolized at every level, and
19
+ # empty when it has none
20
+ attr_reader :config
21
+ # @return [Array<String>] the labels this version carries, such as `"production"`
22
+ attr_reader :labels
23
+ # @return [Array<String>]
24
+ attr_reader :tags
6
25
 
26
+ # @api private
7
27
  VARIABLE_REGEX = /\{\{([a-zA-Z0-9_]+)\}\}/
8
28
 
29
+ # @api private
9
30
  def initialize(name:, type:, prompt:, version:, **extra_args)
10
31
  @name = name
11
32
  @type = type
@@ -16,9 +37,14 @@ module NitroIntelligence
16
37
  @tags = extra_args[:tags] || []
17
38
  end
18
39
 
19
- # Returns prompt "content" from API with prompt variables replaced
20
- # Prompt "content" will either be a string or an array of hashes
21
- # based on prompt "type" ("text" or "chat")
40
+ # The prompt's content with its `{{variables}}` filled in. A variable with no replacement is
41
+ # left as it is.
42
+ #
43
+ # @param replacements [Hash{Symbol => #to_s}] each variable's value, by name
44
+ # @return [String, Array<Hash>] a string for a text prompt, or new messages for a chat prompt,
45
+ # leaving this prompt's own uncompiled
46
+ # @example
47
+ # client.prompt("Page Finder Search").compile(product: "Nitro")
22
48
  def compile(**replacements)
23
49
  return replace_variables(@prompt, **replacements) if @type == "text"
24
50
 
@@ -28,6 +54,8 @@ module NitroIntelligence
28
54
  end
29
55
  end
30
56
 
57
+ # @api private
58
+ #
31
59
  # Takes provided chat messages and inserts the compiled prompt
32
60
  # into the correct position based on prompt "type" ("text" or "chat")
33
61
  def interpolate(messages:, variables:)
@@ -38,6 +66,9 @@ module NitroIntelligence
38
66
  end
39
67
  end
40
68
 
69
+ # The names of the `{{variables}}` the prompt's content uses, in order of appearance.
70
+ #
71
+ # @return [Array<Symbol>]
41
72
  def variables
42
73
  messages = @type == "text" ? [@prompt] : @prompt.pluck(:content)
43
74
 
@@ -0,0 +1,47 @@
1
+ module NitroIntelligence
2
+ # What {Client::Base#rerank} returns: the documents' relevance to the query, most relevant first.
3
+ class Ranking
4
+ # One document's place in the ranking.
5
+ #
6
+ # @!attribute [r] index
7
+ # @return [Integer] the document's position in the `documents` given to {Client::Base#rerank}
8
+ # @!attribute [r] relevance_score
9
+ # @return [Float, nil] how relevant the document is to the query, from 0 to 1, or nil when the
10
+ # provider sent no score, which ranks the document last
11
+ Result = Data.define(:index, :relevance_score)
12
+
13
+ # @return [String, nil] the gateway's identifier for the request
14
+ attr_reader :id
15
+ # @return [Array<Result>] the documents ranked, most relevant first, limited to `top_n` when
16
+ # that was given
17
+ attr_reader :results
18
+ # @return [ActiveSupport::HashWithIndifferentAccess] what the gateway reported as billed:
19
+ # `tokens` (`input_tokens`) and `billed_units` (`search_units`, `total_tokens`), each where the
20
+ # provider sent them
21
+ attr_reader :meta
22
+ # @api private
23
+ attr_reader :response
24
+
25
+ # @!method last_response
26
+ # @return [OpenAI::ResponseMetadata, nil] the HTTP response, including the
27
+ # gateway's headers
28
+ delegate :last_response, to: :response
29
+
30
+ # @api private
31
+ # @param top_n [Integer, nil] how many results to keep, applied here because not every
32
+ # provider honours it
33
+ def initialize(response, top_n: nil)
34
+ @response = response
35
+ @id = response[:id]
36
+ @results = response[:results].to_a.map do |result|
37
+ Result.new(index: result[:index], relevance_score: Float(result[:relevance_score], exception: false))
38
+ end
39
+ # Ties keep the order the provider gave them, since sort_by alone is not stable.
40
+ @results = @results.each_with_index.sort_by do |result, index|
41
+ [-(result.relevance_score || -Float::INFINITY), index]
42
+ end.map(&:first)
43
+ @results = @results.first(top_n) if top_n
44
+ @meta = response[:meta].to_h.deep_dup.with_indifferent_access
45
+ end
46
+ end
47
+ end
@@ -1,4 +1,4 @@
1
1
  module NitroIntelligence
2
2
  # The gem's version.
3
- VERSION = "4.2.0".freeze
3
+ VERSION = "4.3.0".freeze
4
4
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: nitro_intelligence
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.2.0
4
+ version: 4.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Igor Artemenko
@@ -108,19 +108,24 @@ files:
108
108
  - lib/nitro_intelligence/client/handlers/base_handler.rb
109
109
  - lib/nitro_intelligence/client/handlers/chat_handler.rb
110
110
  - lib/nitro_intelligence/client/handlers/decision_handler.rb
111
+ - lib/nitro_intelligence/client/handlers/embedding_handler.rb
111
112
  - lib/nitro_intelligence/client/handlers/image_handler.rb
112
113
  - lib/nitro_intelligence/client/handlers/observed/audio_transcription_handler.rb
113
114
  - lib/nitro_intelligence/client/handlers/observed/base_handler.rb
114
115
  - lib/nitro_intelligence/client/handlers/observed/chat_handler.rb
115
116
  - lib/nitro_intelligence/client/handlers/observed/decision_handler.rb
117
+ - lib/nitro_intelligence/client/handlers/observed/embedding_handler.rb
116
118
  - lib/nitro_intelligence/client/handlers/observed/image_handler.rb
119
+ - lib/nitro_intelligence/client/handlers/observed/rerank_handler.rb
117
120
  - lib/nitro_intelligence/client/handlers/observed/text_to_speech_handler.rb
121
+ - lib/nitro_intelligence/client/handlers/rerank_handler.rb
118
122
  - lib/nitro_intelligence/client/handlers/text_to_speech_handler.rb
119
123
  - lib/nitro_intelligence/client/observed.rb
120
124
  - lib/nitro_intelligence/client/observers/langfuse_observer.rb
121
125
  - lib/nitro_intelligence/configuration.rb
122
126
  - lib/nitro_intelligence/decision.rb
123
127
  - lib/nitro_intelligence/deprecator.rb
128
+ - lib/nitro_intelligence/embeddings.rb
124
129
  - lib/nitro_intelligence/langfuse_extension.rb
125
130
  - lib/nitro_intelligence/langfuse_tracer_provider.rb
126
131
  - lib/nitro_intelligence/media/audio.rb
@@ -142,6 +147,7 @@ files:
142
147
  - lib/nitro_intelligence/observation.rb
143
148
  - lib/nitro_intelligence/pending_review.rb
144
149
  - lib/nitro_intelligence/railtie.rb
150
+ - lib/nitro_intelligence/ranking.rb
145
151
  - lib/nitro_intelligence/reporter.rb
146
152
  - lib/nitro_intelligence/review_decision.rb
147
153
  - lib/nitro_intelligence/run_result.rb