embedding_util 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 41974235f403d3694132565fa69ecf07ca338789bd197c34e2563eb7bd38ad06
+  data.tar.gz: c05864e6cda0f5881d44d676ab8b8cb7dc9d7ceee70aff8394a3ee79f8166a69
+SHA512:
+  metadata.gz: b3500877638960c73de3ebf04ef07441392b86f22108014e4f27e872b95d9ff52cf1850e423443c51bab16ea4c5149a851f619faa53d7f44480f99ae7ebdd759
+  data.tar.gz: e55f3761208466504a9865115d23e0f1ab68bfb146a08ddfcb1c2ae7df4b37a859a80d99aa52139bf6b005fa6ddf4624a96046d58c80f923d183316234042232

data/.rspec

@@ -0,0 +1,2 @@
+--require spec_helper
+--format documentation

data/.rubocop.yml

@@ -0,0 +1,27 @@
+AllCops:
+  TargetRubyVersion: 3.2
+  NewCops: disable
+
+Style/StringLiterals:
+  EnforcedStyle: double_quotes
+  Exclude:
+    - "spec/**/*"
+    - "embedding_util.gemspec"
+
+Style/FrozenStringLiteralComment:
+  Enabled: false
+
+Style/Documentation:
+  Enabled: false
+
+Layout/LineLength:
+  Max: 170
+
+Metrics/BlockLength:
+  Enabled: false
+Metrics/ClassLength:
+  Enabled: false
+Metrics/MethodLength:
+  Enabled: false
+Metrics/AbcSize:
+  Enabled: false

data/CHANGELOG.md

@@ -0,0 +1,8 @@
+## [0.1.0] - 2026-06-08
+
+- Initial release
+- Add local-first embedding and true reranking API
+- Add pinned `small_multilingual_v1` Qwen3 embedding/reranker profile
+- Add endpoint provider for llama.cpp-compatible embedding/reranking APIs
+- Add self-hosted local server management through Ramalama or direct `llama-server`
+- Add `embedding_util` CLI with `support`, `profiles`, `embed`, `rerank`, and `serve`

data/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 hmdne
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,176 @@
+# embedding_util
+
+Local-first text embeddings and reranking for Ruby.
+
+`embedding_util` provides a small require-and-use API for computing embedding vectors and true reranking scores through local model runtimes. It can start local model servers on demand, or use explicitly configured llama.cpp/Ramalama-compatible HTTP endpoints.
+
+The default model profile is `small_multilingual_v1`, pinned to Qwen3 0.6B GGUF embedding and reranker models.
+
+This gem is in the `0.x` series. The API is intentionally unstable until `1.0`, and public method names, configuration options, return shapes, and default profiles may change between minor releases.
+
+## Installation
+
+Add the gem to your Gemfile:
+
+```ruby
+gem "embedding_util"
+```
+
+Then install dependencies:
+
+```sh
+bundle install
+```
+
+## Quick Start
+
+Install either `ramalama` or `llama-server`. Ramalama is preferred; direct `llama-server` is used when Ramalama is unavailable.
+
+Then call the Ruby API:
+
+```ruby
+require "embedding_util"
+
+vector = EmbeddingUtil.embed("hello world")
+ranked = EmbeddingUtil.rerank("Which document is about software?", [
+  "Ruby is a programming language.",
+  "Fresh bread is often served warm.",
+  "A command-line tool can automate repetitive work."
+])
+```
+
+With the default configuration, `EmbeddingUtil.embed` and `EmbeddingUtil.rerank` reuse an already-running local server when one is available. If not, they start the required local model server automatically. Auto-started servers bind to `127.0.0.1`, prefer ports `18080` for embeddings and `18081` for reranking, and choose the next free local port if the preferred port is already in use. First-time use downloads the pinned `small_multilingual_v1` GGUF files through the selected runtime.
+
+## Application-Managed Servers
+
+Applications can manage model servers themselves instead of using automatic self-hosting. This is useful when the embedding/reranking servers run as separate processes, system services, containers, or on another machine.
+
+Configure the endpoints explicitly:
+
+```ruby
+require "embedding_util"
+
+EmbeddingUtil.configure do |config|
+  config.embedding_endpoint = "http://embedding.internal:18080"
+  config.reranker_endpoint = "http://reranker.internal:18081"
+end
+
+vector = EmbeddingUtil.embed("hello world")
+```
+
+Configured endpoints take precedence over automatic self-hosting.
+
+`embedding_util serve` is one convenient way to run compatible servers yourself, but it is optional. You can also run Ramalama, direct `llama-server`, containers, or service units independently as long as they expose llama.cpp-compatible embedding/reranking HTTP APIs.
+
+```sh
+embedding_util serve --model embedding-small_multilingual_v1
+embedding_util serve --model reranker-small_multilingual_v1
+```
+
+`serve` starts one model server per command and runs until stopped. Add `--shutdown-idle SECONDS` only when you want that manually managed server to stop itself after idle output; omit it, set it to `nil`, or pass `0` to disable idle shutdown.
+
+## CLI
+
+```sh
+embedding_util support
+embedding_util profiles
+embedding_util serve --model embedding-small_multilingual_v1
+embedding_util embed "hello world"
+embedding_util embed "hello world" --verbose
+embedding_util rerank \
+  "Which document is about software?" \
+  "Ruby is a programming language." \
+  "Fresh bread is often served warm." \
+  "A command-line tool can automate repetitive work."
+```
+
+`embed` prints a JSON array. `rerank` prints JSON objects with `index`, `document`, `score`, and `metadata`.
+
+`serve` starts one local model server. The default model is `embedding-small_multilingual_v1`; use `reranker-small_multilingual_v1` for the reranker server. By default, `serve` uses Ramalama when available and falls back to direct `llama-server`. It runs until stopped unless a positive `--shutdown-idle` value is provided.
+
+Explicit `serve --port PORT` requires that exact port to be free. Without `--port`, `serve` prefers the profile default port and chooses the next free local port if needed.
+
+Use `--verbose` on `embed` or `rerank` to print self-hosting diagnostics, including the background `serve` command and log path. First-time model downloads are expected to work with the default startup timeout; use `--startup-timeout` only when you explicitly want to shorten or extend that wait.
+
+## API
+
+- `EmbeddingUtil.embed(text)` returns one embedding array.
+- `EmbeddingUtil.embed_many(texts)` returns one embedding array per input text.
+- `EmbeddingUtil.embed_result(text_or_texts)` returns embeddings plus provider/model metadata.
+- `EmbeddingUtil.rerank(query, documents)` returns ranked `EmbeddingUtil::RankedDocument` objects.
+- `EmbeddingUtil.rerank_result(query, documents)` returns ranked documents plus provider/model metadata.
+- `EmbeddingUtil.support` reports configured provider support.
+- `EmbeddingUtil.profiles` returns known immutable model profiles.
+- `embedding_util support`, `profiles`, `embed`, `rerank`, and `serve` expose the same local-first behavior from the command line.
+
+## Default Profile
+
+`small_multilingual_v1` is intentionally pinned because embedding vectors are model-output-specific.
+
+Embedding model:
+
+- repo: `Qwen/Qwen3-Embedding-0.6B-GGUF`
+- file: `Qwen3-Embedding-0.6B-Q8_0.gguf`
+- dimensions: `1024`
+- server flags: `--embedding --pooling last`
+
+Reranker model:
+
+- repo: `ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF`
+- file: `qwen3-reranker-0.6b-q8_0.gguf`
+- server flags: `--reranking`
+
+Do not combine embedding and reranking flags for this profile. Run separate local servers.
+
+## Configuration
+
+```ruby
+EmbeddingUtil.configure do |config|
+  config.profile = :small_multilingual_v1
+  config.runtime = :auto
+  config.host = "127.0.0.1"
+  config.embedding_port = 18080
+  config.reranker_port = 18081
+  config.startup_timeout = 3600
+  config.shutdown_idle = 300
+  config.timeout = 60
+end
+```
+
+Explicit local endpoints can still be configured when you manage servers yourself:
+
+```ruby
+EmbeddingUtil.configure do |config|
+  config.embedding_endpoint = "http://127.0.0.1:18080"
+  config.reranker_endpoint = "http://127.0.0.1:18081"
+end
+```
+
+Environment variables are also supported:
+
+- `EMBEDDING_UTIL_ENDPOINT` for one endpoint serving both APIs
+- `EMBEDDING_UTIL_EMBEDDING_ENDPOINT`
+- `EMBEDDING_UTIL_RERANKER_ENDPOINT`
+- `EMBEDDING_UTIL_TIMEOUT`
+- `EMBEDDING_UTIL_STARTUP_TIMEOUT`
+- `EMBEDDING_UTIL_RUNTIME`
+- `EMBEDDING_UTIL_SHUTDOWN_IDLE`
+- `EMBEDDING_UTIL_STATE_DIR`
+- `EMBEDDING_UTIL_VERBOSE`
+- `EMBEDDING_UTIL_EMBEDDING_PORT`
+- `EMBEDDING_UTIL_RERANKER_PORT`
+
+## Development
+
+```sh
+bundle install
+bundle exec rake
+```
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at `https://github.com/rbutils/embedding_util`.
+
+## License
+
+The gem is available as open source under the terms of the MIT License.

data/Rakefile

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+require "rubocop/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]

data/exe/embedding_util

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "embedding_util"
+
+EmbeddingUtil::CLI.start(ARGV)

data/lib/embedding_util/cli.rb

@@ -0,0 +1,111 @@
+# frozen_string_literal: true
+
+require "json"
+require "thor"
+
+module EmbeddingUtil
+  class CLI < Thor
+    CONFIG_OPTIONS = {
+      profile: :to_sym.to_proc,
+      runtime: ->(value) { RuntimeCommand.normalize_runtime(value) },
+      endpoint: ->(value) { value },
+      embedding_endpoint: ->(value) { value },
+      reranker_endpoint: ->(value) { value },
+      timeout: ->(value) { value },
+      startup_timeout: ->(value) { value },
+      shutdown_idle: :to_i.to_proc,
+      verbose: ->(value) { value }
+    }.freeze
+
+    class_option :endpoint, type: :string, desc: "Endpoint serving both embedding and reranking APIs"
+    class_option :embedding_endpoint, type: :string, desc: "Endpoint serving /v1/embeddings"
+    class_option :reranker_endpoint, type: :string, desc: "Endpoint serving /v1/rerank or /rerank"
+    class_option :profile, type: :string, desc: "Model profile"
+    class_option :runtime, type: :string, desc: "Self-hosting runtime: auto, ramalama, or llama-server"
+    class_option :timeout, type: :numeric, desc: "HTTP timeout in seconds"
+    class_option :startup_timeout, type: :numeric, desc: "Seconds to wait for self-hosted server startup"
+    class_option :shutdown_idle, type: :numeric, desc: "Stop self-hosted server after this many seconds without stdout/stderr activity"
+    class_option :verbose, type: :boolean, desc: "Print self-hosting diagnostics"
+
+    desc "support", "Display configured provider support"
+    def support
+      configure_embedding_util
+      EmbeddingUtil.support.each do |item|
+        status = item.fetch(:supported) ? "supported" : "not supported"
+        puts "#{item.fetch(:provider)}: #{status}"
+        puts "  embedding_endpoint: #{item.fetch(:embedding_endpoint)}" if item[:embedding_endpoint]
+        puts "  reranker_endpoint: #{item.fetch(:reranker_endpoint)}" if item[:reranker_endpoint]
+      end
+    end
+
+    desc "profiles", "List known model profiles"
+    def profiles
+      EmbeddingUtil.profiles.each do |profile|
+        puts profile.name
+        puts "  embedding: #{profile.embedding.fetch(:repo)} / #{profile.embedding.fetch(:file)}"
+        puts "  reranker:  #{profile.reranker.fetch(:repo)} / #{profile.reranker.fetch(:file)}"
+      end
+    end
+
+    desc "embed TEXT", "Compute one embedding and print it as JSON"
+    def embed(text)
+      configure_embedding_util
+      puts JSON.generate(EmbeddingUtil.embed(text))
+    rescue Error => e
+      abort e.message
+    end
+
+    desc "rerank QUERY DOCUMENT...", "Rerank documents and print ranked results as JSON"
+    def rerank(query, *documents)
+      configure_embedding_util
+      raise Error, "provide at least one document to rerank" if documents.empty?
+
+      results = EmbeddingUtil.rerank(query, documents).map do |result|
+        {
+          index: result.index,
+          document: result.document,
+          score: result.score,
+          metadata: result.metadata
+        }
+      end
+      puts JSON.pretty_generate(results)
+    rescue Error => e
+      abort e.message
+    end
+
+    desc "serve", "Start one local model server and stop it after stdout/stderr is idle"
+    option :model, type: :string, default: "embedding-small_multilingual_v1",
+                   desc: "Model server to run, such as embedding-small_multilingual_v1 or reranker-small_multilingual_v1"
+    option :port, type: :numeric, desc: "Port for the model server"
+    option :host, type: :string, default: "127.0.0.1", desc: "Host for the model server"
+    def serve
+      configure_embedding_util
+      ServerManager.new(config: EmbeddingUtil.configuration).serve(
+        model: options[:model],
+        runtime: options[:runtime] || EmbeddingUtil.configuration.runtime,
+        shutdown_idle: options[:shutdown_idle]&.to_i,
+        host: options[:host],
+        port: options[:port]&.to_i
+      )
+    end
+
+    no_commands do
+      def configure_embedding_util
+        EmbeddingUtil.configure do |config|
+          cli_config.each do |key, value|
+            config.public_send("#{key}=", value)
+          end
+        end
+      end
+
+      def cli_config
+        CONFIG_OPTIONS.each_with_object({}) do |(key, coercion), values|
+          value = options[key]
+          next if value.nil?
+
+          values[key] = coercion.call(value)
+        end
+      end
+    end
+  end
+end

data/lib/embedding_util/configuration.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module EmbeddingUtil
+  class Configuration
+    attr_accessor :profile, :provider, :endpoint, :embedding_endpoint, :reranker_endpoint, :timeout, :startup_timeout, :shutdown_idle, :host,
+                  :embedding_port, :reranker_port, :state_dir, :verbose
+    attr_reader :runtime
+
+    def initialize
+      @profile = :small_multilingual_v1
+      @provider = :auto
+      self.runtime = ENV.fetch("EMBEDDING_UTIL_RUNTIME", "auto")
+      @endpoint = ENV["EMBEDDING_UTIL_ENDPOINT"]
+      @embedding_endpoint = ENV["EMBEDDING_UTIL_EMBEDDING_ENDPOINT"]
+      @reranker_endpoint = ENV["EMBEDDING_UTIL_RERANKER_ENDPOINT"]
+      @timeout = Float(ENV.fetch("EMBEDDING_UTIL_TIMEOUT", "60"))
+      @startup_timeout = Float(ENV.fetch("EMBEDDING_UTIL_STARTUP_TIMEOUT", "3600"))
+      @shutdown_idle = Integer(ENV.fetch("EMBEDDING_UTIL_SHUTDOWN_IDLE", "300"))
+      @host = ENV.fetch("EMBEDDING_UTIL_HOST", "127.0.0.1")
+      @embedding_port = Integer(ENV.fetch("EMBEDDING_UTIL_EMBEDDING_PORT", "18080"))
+      @reranker_port = Integer(ENV.fetch("EMBEDDING_UTIL_RERANKER_PORT", "18081"))
+      @state_dir = ENV.fetch("EMBEDDING_UTIL_STATE_DIR", File.expand_path("~/.local/state/embedding_util"))
+      @verbose = ENV.fetch("EMBEDDING_UTIL_VERBOSE", "false").match?(/\A(?:1|true|yes|on)\z/i)
+    end
+
+    def runtime=(value)
+      @runtime = value.to_s.tr("-", "_").to_sym
+    end
+
+    def resolved_profile
+      profile.is_a?(Profile) ? profile : Profiles.fetch(profile)
+    end
+
+    def embedding_endpoint_url
+      embedding_endpoint || endpoint
+    end
+
+    def reranker_endpoint_url
+      reranker_endpoint || endpoint
+    end
+  end
+end

data/lib/embedding_util/profile.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module EmbeddingUtil
+  Profile = Data.define(:name, :embedding, :reranker) do
+    def initialize(name:, embedding:, reranker:)
+      super(name: name.to_sym, embedding: deep_freeze(embedding), reranker: deep_freeze(reranker))
+    end
+
+    private
+
+    def deep_freeze(value)
+      case value
+      when Hash
+        value.transform_values { |item| deep_freeze(item) }.freeze
+      when Array
+        value.map { |item| deep_freeze(item) }.freeze
+      else
+        value.freeze
+      end
+    end
+  end
+end

data/lib/embedding_util/profiles.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+require_relative "profile"
+
+module EmbeddingUtil
+  module Profiles
+    SMALL_MULTILINGUAL_V1 = Profile.new(
+      name: :small_multilingual_v1,
+      embedding: {
+        repo: "Qwen/Qwen3-Embedding-0.6B-GGUF",
+        file: "Qwen3-Embedding-0.6B-Q8_0.gguf",
+        model: "qwen3-embedding-0.6b",
+        dimensions: 1024,
+        normalize: true,
+        pooling: "last",
+        server_flags: ["--embedding", "--pooling", "last"]
+      },
+      reranker: {
+        repo: "ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF",
+        file: "qwen3-reranker-0.6b-q8_0.gguf",
+        model: "qwen3-reranker-0.6b",
+        server_flags: ["--reranking"]
+      }
+    )
+
+    BY_NAME = {
+      SMALL_MULTILINGUAL_V1.name => SMALL_MULTILINGUAL_V1
+    }.freeze
+
+    module_function
+
+    def fetch(name)
+      BY_NAME.fetch(name.to_sym) do
+        raise ArgumentError, "unknown embedding_util profile: #{name.inspect}"
+      end
+    end
+
+    def all
+      BY_NAME.values
+    end
+  end
+end

data/lib/embedding_util/provider.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module EmbeddingUtil
+  class Provider
+    attr_reader :config
+
+    def initialize(config: EmbeddingUtil.configuration)
+      @config = config
+    end
+
+    def self.provider_name
+      name.split("::").last.gsub(/([a-z\d])([A-Z])/, "\\1_\\2").downcase.to_sym
+    end
+
+    def self.supported?(config = EmbeddingUtil.configuration)
+      new(config: config).supported?
+    end
+
+    def provider_name
+      self.class.provider_name
+    end
+
+    def supported?
+      false
+    end
+
+    def support
+      { provider: provider_name, supported: supported? }
+    end
+  end
+end

data/lib/embedding_util/provider_registry.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+module EmbeddingUtil
+  class ProviderRegistry
+    attr_reader :providers
+
+    def initialize
+      @providers = []
+    end
+
+    def register(provider_class)
+      @providers << provider_class unless @providers.include?(provider_class)
+    end
+
+    def resolve(config: EmbeddingUtil.configuration)
+      selected = config.provider
+      return resolve_selected(selected, config) if selected && selected != :auto
+
+      provider_class = providers.find { |candidate| candidate.supported?(config) }
+      raise UnsupportedProviderError, unsupported_message unless provider_class
+
+      provider_class.new(config: config)
+    end
+
+    def support(config: EmbeddingUtil.configuration)
+      providers.map { |provider_class| provider_class.new(config: config).support }
+    end
+
+    private
+
+    def resolve_selected(selected, config)
+      provider_class = providers.find { |candidate| candidate.provider_name == selected.to_sym }
+      raise UnsupportedProviderError, "unknown embedding_util provider: #{selected.inspect}" unless provider_class
+
+      provider = provider_class.new(config: config)
+      raise UnsupportedProviderError, unsupported_message(selected) unless provider.supported?
+
+      provider
+    end
+
+    def unsupported_message(provider = nil)
+      target = provider ? "provider #{provider.inspect}" : "a supported local embedding provider"
+      <<~MESSAGE.strip
+        Could not find #{target}.
+
+        Configure already-running local endpoints:
+          EmbeddingUtil.configure { |c| c.embedding_endpoint = "http://127.0.0.1:18080" }
+          EmbeddingUtil.configure { |c| c.reranker_endpoint = "http://127.0.0.1:18081" }
+
+        Or install a local runtime for self-hosting:
+          ramalama
+          llama-server
+      MESSAGE
+    end
+  end
+end