ruby_decision_model 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f1c6972a68e2a0545795663a4c792603a692eb65b609d630ce2754e9f032d6b2
-  data.tar.gz: 4252b768aa5f7ab6e16eee20ef09ca16923fdf2b192eb21a0fb79f06f51f8b42
+  metadata.gz: 0d92e5f4d5ae068f244d1adb69f59f536361046d28fe58923f0b347650122df1
+  data.tar.gz: 8e6a26811a717449fc52e4c53f076931fa1998fa28e4ec9937df27eff3bfd0f6
 SHA512:
-  metadata.gz: 69abd24fe4a39391791c183b0880a8c909e8521a5155c7d1ac92be380780572a5d23423ed32cb9b1267f2f110fbe9a9958982dca5edd2486f4b43a9e3e0cb1d6
-  data.tar.gz: 40d47a12bf922d452ad350180d2d1fbac189a16f2a732f3885784a7f71438e30a7e8dd7b75b875e5eeb8882ffc36a22c214177c287d13622a329322edc1b6030
+  metadata.gz: c100363ee1b8f540916da3e32a051e3de427dcb5625a63254604cc5388f9059aa2df74497d3a1a449d4a6121f366be36de0b548b03966f95616b65d07e0d3723
+  data.tar.gz: efcc1c0abf9412978ca89dc8777c91950d9cf90600b1bc6444d51cf8fedde10b5b9399b9af16105821f3773d28ae2905fed30563558faefd8e50bdd9ee981146

data/CHANGELOG.md

@@ -1,5 +1,25 @@
 # Changelog
 
+## 0.3.0 - 2026-10-12
+
+Cloudflare's `clef-omni` reads audio and video.
+
+- `Client#ask` takes `audio:` and `videos:`, arrays of base64 data URLs, for
+  `clef-omni`. Cloudflare reads WAV or MP3 audio and MP4 or WebM video. Added
+  `RubyDecisionModel::Media.data_url` and `.from_file`, which build data URLs
+  for images, audio, and video.
+- When the provider or model doesn't read audio or video, including `clef`
+  and `clef-flash`, the client raises `RequestError` before sending.
+- `cloudflare/clef-omni` and `@cf/cloudflare/clef-omni` resolve to
+  `clef-omni` on the Cloudflare provider. In 0.2.0 they raised
+  `ConfigurationError`. The bare `clef-omni` already worked.
+- New provider hooks `supports_audio?(model)` and `supports_videos?(model)`,
+  both false by default. `Base#request_body` accepts `audio:` and `videos:`.
+- `rake smoke` takes `SMOKE_AUDIO` and `SMOKE_VIDEO`.
+- Hosted `clef-flash` now has a 24,576-token context window. Cloudflare cuts
+  a text state that runs past the window to fit instead of rejecting it. The
+  gem is unchanged.
+
 ## 0.2.0 - 2026-10-06
 
 The decision models that shipped after Jev, behind the same `Client`.

data/README.md

@@ -127,7 +127,9 @@ client = RubyDecisionModel::Client.new(provider: provider)
 
 Requests go to
 `https://api.cloudflare.com/client/v4/accounts/<account_id>/ai/run/@cf/cloudflare/<model>`.
-The default model is `clef`; `clef-flash` is the smaller, faster one. The body
+The default model is `clef`; `clef-flash` is the smaller, faster one, and
+`clef-omni` also reads audio and video (see
+[Images, audio, and video](#images-audio-and-video)). The body
 is System One. The provider unwraps Cloudflare's `{"result": ...}` envelope,
 and a body with `"success": false` raises `InvalidResponse` carrying
 Cloudflare's error text. The account id and model end up in the URL path, so
@@ -138,6 +140,11 @@ raises `ConfigurationError`.
 Usage reports tokens with no cost, and `response.request_id` is the `cf-ray`
 header.
 
+Hosted `clef-flash` has a 24,576-token context window; `clef` and `clef-omni`
+have about 64k. Cloudflare cuts a text state that runs past the window to
+fit and returns no error. Images, audio, and video that don't fit fail the
+request.
+
 ### Perplexity (pplx-decider)
 
 ```ruby
@@ -206,7 +213,9 @@ Databricks, and System One servers default to 30, since Perplexity documents
 responses of up to 23 seconds on large inputs and a local server may load the
 model on the first request. Connecting gets 5 seconds either way. A number
 passed as `timeout:` sets both, as in 0.1.0. Through OpenRouter, pass a
-longer `timeout:` yourself for large inputs to slower models.
+longer `timeout:` yourself for large inputs to slower models. Audio and
+video take longer than text: Cloudflare reports about 1.5 seconds for a
+21-second video with sound. Pass a longer `timeout:` for long clips.
 
 ### Model aliases
 
@@ -225,26 +234,31 @@ whatever the provider returned.
 | `"luna"`, `"gpt-6-luna"` | `openai/gpt-6-luna-decisions` | `gpt-6-luna` (OpenAI) |
 | `"openai/gpt-6-luna-decisions"`, `"openai/gpt-6-luna"`, `"gpt-6-luna-decisions"` | as given | `gpt-6-luna` (OpenAI) |
 | `"clef"`, `"clef-flash"` | `cloudflare/clef`, `cloudflare/clef-flash` | as given (Cloudflare) |
-| `"cloudflare/clef"`, `"cloudflare/clef-flash"` | as given | `clef`, `clef-flash` (Cloudflare) |
-| `"@cf/cloudflare/clef"`, `"@cf/cloudflare/clef-flash"` | as given | `clef`, `clef-flash` (Cloudflare) |
+| `"cloudflare/clef"`, `"cloudflare/clef-flash"`, `"cloudflare/clef-omni"` | as given | `clef`, `clef-flash`, `clef-omni` (Cloudflare) |
+| `"@cf/cloudflare/clef"`, `"@cf/cloudflare/clef-flash"`, `"@cf/cloudflare/clef-omni"` | as given | `clef`, `clef-flash`, `clef-omni` (Cloudflare) |
 | `"pplx-decider"` | `perplexity/pplx-decider-v1-27b` | `pplx-decider-v1.1-27b` (Perplexity) |
 | `"pplx-decider-v1-27b"` | `perplexity/pplx-decider-v1-27b` | as given (Perplexity) |
 | `"perplexity/pplx-decider-v1-27b"` | as given | `pplx-decider-v1-27b` (Perplexity) |
 | anything else | as given | as given |
 
 OpenRouter carries pplx-decider v1 only, so `"pplx-decider"` means v1 there
-and v1.1 on Perplexity's own API.
+and v1.1 on Perplexity's own API. OpenRouter doesn't list `clef-omni` as of
+October 2026, so it has no alias there; on Cloudflare pass `"clef-omni"`.
 
 ### Writing a provider
 
 Subclass `RubyDecisionModel::Providers::Base` and define `name`, `env_var`,
 `default_base_url`, `endpoint_path`, and `default_model`. Optional hooks:
-`aliases`, `reports_cost?`, `supports_images?`, `request_id_header`,
+`aliases`, `reports_cost?`, `supports_images?`, `supports_audio?(model)`,
+`supports_videos?(model)`, `request_id_header`,
 `requires_api_key?`, `default_timeout`, `error_message`, and `validate!`. When the wire format differs from
 System One, override `request_body` to encode and `normalize_response` to turn
 the parsed body back into System One answers keyed by question id; the
 OpenAI provider shows both directions. Override `url(model)` when the model
-belongs in the path. Pass an instance as `provider:`.
+belongs in the path. A provider that answers true from `supports_audio?` or
+`supports_videos?` gets `audio:` or `videos:` passed to `request_body`, so
+an override of `request_body` has to accept them. Pass an instance as
+`provider:`.
 
 ## Questions and answers
 
@@ -268,7 +282,7 @@ approximately 1; treat them as calibrated, not normalized. Noul
 `probabilities` are always `{"true" => p, "false" => 1 - p}`. Most providers
 send only the probability, and the client fills in the split.
 
-## Images
+## Images, audio, and video
 
 OpenAI, Cloudflare, Perplexity, and System One servers read images. Pass them
 as base64 data URLs; no provider fetches a remote URL.
@@ -290,6 +304,39 @@ OpenAI, the `images` field for Cloudflare and System One servers, and
 Databricks don't take images, so the client raises `RequestError` before
 sending. Size and count limits vary by vendor and are enforced server side.
 
+Cloudflare's `clef-omni` also reads audio (WAV or MP3) and video (MP4 or
+WebM), passed as `audio:` and `videos:`. `RubyDecisionModel::Media.from_file`
+and `.data_url` build data URLs for all three kinds, and `Images` still works
+for images.
+
+```ruby
+client = RubyDecisionModel::Client.new(provider: :cloudflare, model: "clef-omni")
+
+client.ask(
+  state: "Customer sent a photo, a voice note, and a short video of the fault.",
+  questions: { "faulty" => RubyDecisionModel::Questions.noul("Is the device faulty?") },
+  images: [RubyDecisionModel::Media.from_file("device.jpg")],
+  audio: [RubyDecisionModel::Media.from_file("note.mp3")],
+  videos: [RubyDecisionModel::Media.from_file("fault.mp4")]
+)
+```
+
+On Cloudflare they go in the `images`, `audio`, and `videos` fields. A
+provider or model that doesn't read audio or video, including `clef` and
+`clef-flash`, makes the client raise `RequestError` before sending.
+Cloudflare documents up to 4 images, 4 audio clips, and 2 videos per request,
+enforced server side.
+
+The client checks only that each item is a data URL of the right kind
+(`data:image/`, `data:audio/`, or `data:video/`). It does not decode the
+payload or measure it, and `from_file` reads the whole file into memory.
+Format, size, and count are the provider's to enforce, so check uploads you
+don't control before you pass them on.
+
+A retry sends the whole body again, so with the default policy a request
+carrying large clips can upload up to three times. Build the client with
+`retry: { max_retries: 0 }` to turn retries off for those calls.
+
 ## Retries
 
 Retry behaviour follows the official Typesafe SDKs and lives in
@@ -338,7 +385,7 @@ return is still accepted and treated as having no headers, which means no
 | Error | Meaning |
 | --- | --- |
 | `ConfigurationError` | No provider could be resolved, a missing key or setting (api_key, account_id, base_url), unknown provider, a `model:` Databricks can't take, or bad `retry:` value |
-| `RequestError` | Questions hash was empty, images were not data URLs or went to a provider that doesn't read them, or state or questions could not be encoded as JSON (original error on `#cause`) |
+| `RequestError` | Questions hash was empty, images, audio, or videos were not data URLs or went to a provider or model that doesn't read them, or state or questions could not be encoded as JSON (original error on `#cause`) |
 | `TransportError` (`TimeoutError`) | Network or timeout failure after retries, carries `#cause_error` |
 | `ApiError` | Non-2xx response, carries `#status`, `#body`, and `#headers`. The message ends with the vendor's own reason when the body has one |
 | `Unauthorized` | 401 |
@@ -349,7 +396,7 @@ return is still accepted and treated as having no headers, which means no
 | `InvalidResponse` | Body wasn't JSON, wasn't a Hash, or an answer was malformed (including a noul outside 0..1 or an id answered twice) |
 | `MissingAnswers` | One or more question ids came back missing, wrong-typed, or refused. Carries `#missing`, `#refused` (the subset the provider declined), and `#answers` |
 
-Status: 0.2.0, API may change.
+Status: 0.3.0, API may change.
 
 The companion gem `decide` builds decisions and verdicts on top of this client.
 
@@ -364,6 +411,8 @@ anywhere. To ship a version:
    request per configured provider and prints the answers. CI never runs it.
    `TARGETS=open_router:clef,open_router:luna` picks provider and model
    pairs, and `SMOKE_IMAGE=photo.png` adds an image where supported.
+   `SMOKE_AUDIO=clip.mp3` and `SMOKE_VIDEO=clip.mp4` add audio and video
+   where the model reads them, such as `TARGETS=cloudflare:clef-omni`.
 4. Merge to `main`. The Release workflow runs the suite, builds the gem with
    `gem build --strict`, checks the built gem carries every file under
    `lib/`, and pushes it. A version already on RubyGems is skipped, so the

data/lib/ruby_decision_model/client.rb

@@ -60,12 +60,13 @@ module RubyDecisionModel
       @provider.base_url
     end
 
-    # images: data URLs (see Images) for providers that read images. Each
-    # provider places them where its API expects.
-    def ask(state:, questions:, images: nil)
+    # images:, audio:, videos: data URLs (see Images and Media) for providers
+    # and models that read them. Each provider places them where its API
+    # expects.
+    def ask(state:, questions:, images: nil, audio: nil, videos: nil)
       raise RequestError, "questions must not be empty" if questions.nil? || questions.empty?
 
-      body = build_body(state, questions, images)
+      body = build_body(state, questions, images: images, audio: audio, videos: videos)
       status, response_body, response_headers = perform_with_retry(
         url: request_url, headers: @provider.headers, body: body
       )
@@ -81,25 +82,44 @@ module RubyDecisionModel
       takes_model ? @provider.url(@model) : @provider.url
     end
 
-    def build_body(state, questions, images)
-      images = Array(images)
-      return @provider.request_body(model: @model, state: state, questions: questions) if images.empty?
+    MEDIA_PREFIXES = { images: "data:image/", audio: "data:audio/", videos: "data:video/" }.freeze
+    private_constant :MEDIA_PREFIXES
 
-      raise RequestError, "#{@provider.name} does not accept images" unless @provider.supports_images?
+    def build_body(state, questions, **media)
+      media = media.transform_values { |urls| Array(urls) }.reject { |_kind, urls| urls.empty? }
 
-      images.each do |image|
-        next if image.is_a?(String) && image.start_with?("data:image/")
+      media.each do |kind, urls|
+        raise RequestError, unsupported_message(kind) unless accepts?(kind)
 
-        raise RequestError, "images must be data URLs (data:image/...); see RubyDecisionModel::Images"
+        prefix = MEDIA_PREFIXES.fetch(kind)
+        next if urls.all? { |url| url.is_a?(String) && url.start_with?(prefix) }
+
+        raise RequestError, "#{kind} must be data URLs (#{prefix}...); see RubyDecisionModel::#{kind == :images ? 'Images' : 'Media'}"
       end
 
-      @provider.request_body(model: @model, state: state, questions: questions, images: images)
+      @provider.request_body(model: @model, state: state, questions: questions, **media)
     rescue JSON::GeneratorError, JSON::NestingError => e
       # The generator's message can quote the offending value, so it stays
       # on #cause rather than in a message that may be logged.
       raise RequestError, "state or questions could not be encoded as JSON (#{e.class})"
     end
 
+    # supports_images? takes no argument in 0.2.0 providers, so it is called
+    # bare; audio and videos can differ per model, so those get the model.
+    def accepts?(kind)
+      case kind
+      when :images then @provider.supports_images?
+      when :audio then @provider.supports_audio?(@model)
+      when :videos then @provider.supports_videos?(@model)
+      end
+    end
+
+    def unsupported_message(kind)
+      return "#{@provider.name} does not accept images" if kind == :images
+
+      "#{[@provider.name, @model].compact.join(' ')} does not accept #{kind}"
+    end
+
     def resolve_provider(provider, api_key:, base_url:)
       case provider
       when Providers::Base

data/lib/ruby_decision_model/media.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+require_relative "images"
+
+module RubyDecisionModel
+  # Builds the base64 data URLs that Client#ask takes as images:, audio:, and
+  # videos:. Images has the image-only version of the same helpers.
+  module Media
+    CONTENT_TYPES = Images::CONTENT_TYPES.merge(
+      ".wav" => "audio/wav",
+      ".mp3" => "audio/mpeg",
+      ".mp4" => "video/mp4",
+      ".webm" => "video/webm"
+    ).freeze
+
+    module_function
+
+    def data_url(bytes, content_type:)
+      unless content_type.to_s.start_with?("image/", "audio/", "video/")
+        raise ArgumentError, "content_type must be an image/*, audio/*, or video/* type, got #{content_type.inspect}"
+      end
+
+      # pack("m0") is strict base64 without the base64 gem, which leaves the
+      # default gems in Ruby 3.4.
+      "data:#{content_type};base64,#{[bytes].pack('m0')}"
+    end
+
+    # Reads a file and infers its type from the extension.
+    def from_file(path, content_type: nil)
+      content_type ||= CONTENT_TYPES[File.extname(path.to_s).downcase]
+      raise ArgumentError, "cannot infer a media type for #{path}; pass content_type:" if content_type.nil?
+
+      data_url(File.binread(path), content_type: content_type)
+    end
+  end
+end

data/lib/ruby_decision_model/providers/base.rb

@@ -65,6 +65,18 @@ module RubyDecisionModel
         false
       end
 
+      # Whether requests may carry audio or videos. Client refuses audio: and
+      # videos: for a provider that says no, before anything goes over the
+      # wire. The model is passed because support can differ between one
+      # provider's models.
+      def supports_audio?(_model = nil)
+        false
+      end
+
+      def supports_videos?(_model = nil)
+        false
+      end
+
       # Response header carrying the provider's request id, or nil.
       # Typesafe's, which 0.1.0 read for every provider and System One
       # servers that follow Typesafe send too. Vendors with their own header
@@ -129,12 +141,14 @@ module RubyDecisionModel
         headers
       end
 
-      # Client passes images: only when the caller supplied some, so a
-      # subclass that does not take images can keep a three-keyword
-      # signature.
-      def request_body(model:, state:, questions:, images: nil)
+      # Client passes images:, audio:, and videos: only when the caller
+      # supplied some and the provider said it supports them, so a subclass
+      # that takes none can keep a three-keyword signature.
+      def request_body(model:, state:, questions:, images: nil, audio: nil, videos: nil)
         body = { "model" => model, "state" => state, "questions" => questions }
         body["images"] = images if images
+        body["audio"] = audio if audio
+        body["videos"] = videos if videos
         JSON.generate(body)
       end
 

data/lib/ruby_decision_model/providers/cloudflare.rb

@@ -4,8 +4,9 @@ require_relative "base"
 
 module RubyDecisionModel
   module Providers
-    # Cloudflare's Clef and Clef-flash on Workers AI. The body is System One
-    # with Cloudflare's images extension; the model also appears in the
+    # Cloudflare's Clef, Clef-flash, and Clef-omni on Workers AI. The body is
+    # System One with Cloudflare's images, audio, and videos extensions
+    # (audio and videos on Clef-omni only); the model also appears in the
     # path, and the account id is required. Responses arrive inside the
     # Workers AI envelope ({"result": ..., "success": true}), which is
     # unwrapped here.
@@ -14,9 +15,14 @@ module RubyDecisionModel
         "cloudflare/clef" => "clef",
         "cloudflare/clef-flash" => "clef-flash",
         "@cf/cloudflare/clef" => "clef",
-        "@cf/cloudflare/clef-flash" => "clef-flash"
+        "@cf/cloudflare/clef-flash" => "clef-flash",
+        "cloudflare/clef-omni" => "clef-omni",
+        "@cf/cloudflare/clef-omni" => "clef-omni"
       }.freeze
 
+      # The models that read audio and videos as well as images.
+      OMNI_MODELS = %w[clef-omni].freeze
+
       # Cloudflare's own tooling reads CLOUDFLARE_API_TOKEN; the Clef docs
       # use CLOUDFLARE_AUTH_TOKEN. Either works, in that order.
       FALLBACK_ENV_VAR = "CLOUDFLARE_AUTH_TOKEN"
@@ -76,6 +82,14 @@ module RubyDecisionModel
         true
       end
 
+      def supports_audio?(model = nil)
+        OMNI_MODELS.include?(model.to_s)
+      end
+
+      def supports_videos?(model = nil)
+        OMNI_MODELS.include?(model.to_s)
+      end
+
       def request_id_header
         "cf-ray"
       end

data/lib/ruby_decision_model/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RubyDecisionModel
-  VERSION = "0.2.0"
+  VERSION = "0.3.0"
 end

data/lib/ruby_decision_model.rb

@@ -7,6 +7,7 @@ require_relative "ruby_decision_model/version"
 require_relative "ruby_decision_model/errors"
 require_relative "ruby_decision_model/questions"
 require_relative "ruby_decision_model/images"
+require_relative "ruby_decision_model/media"
 require_relative "ruby_decision_model/answers"
 require_relative "ruby_decision_model/response"
 require_relative "ruby_decision_model/providers"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ruby_decision_model
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Obie Fernandez
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-10-07 00:00:00.000000000 Z
+date: 2026-10-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -40,11 +40,11 @@ dependencies:
         version: '13.0'
 description: Decision models answer typed questions about a state instead of generating
   text. ruby_decision_model builds noul (yes/no probability), choice, and score questions,
-  posts them with a state and optional images to a decision-model provider (OpenRouter,
-  Typesafe, OpenAI, Cloudflare, Perplexity, Databricks, or any System One server such
-  as Ollama), and returns the same normalized answers with probabilities, confidence,
-  legends, and usage whichever vendor served them. Retries follow the official Typesafe
-  SDKs.
+  posts them with a state and optional images, audio, and video to a decision-model
+  provider (OpenRouter, Typesafe, OpenAI, Cloudflare, Perplexity, Databricks, or any
+  System One server such as Ollama), and returns the same normalized answers with
+  probabilities, confidence, legends, and usage whichever vendor served them. Retries
+  follow the official Typesafe SDKs.
 email:
 - obiefernandez@gmail.com
 executables: []
@@ -59,6 +59,7 @@ files:
 - lib/ruby_decision_model/client.rb
 - lib/ruby_decision_model/errors.rb
 - lib/ruby_decision_model/images.rb
+- lib/ruby_decision_model/media.rb
 - lib/ruby_decision_model/providers.rb
 - lib/ruby_decision_model/providers/base.rb
 - lib/ruby_decision_model/providers/cloudflare.rb