typesafe-ai-ruby 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 21d48754dd7b9582a87e2905f23d01de27c9d8e8f47a14c3eaf5e7f5bccf2e19
-  data.tar.gz: '0958ee9c8ef35da5b53b90dbacfdf3012ceaef7f79c27bf244ff50803deab56c'
+  metadata.gz: 7f892e52635bd8e5f079a2b3f6c9240cff084c3fdb61a11b80e0b4b347708e28
+  data.tar.gz: 00b8ecee6d1afef19586bc64887347a2c0b9b4bfe80b26651c2e0cfaddd147c4
 SHA512:
-  metadata.gz: 6bb1ef123a99b204a3dae6e22ff9567494d328c2ec1bd5ed47061554aa2ac2d2c2b76f796f53cd40cae2f4821d9dee4f7f940b3695b3e34446ef176c922f968c
-  data.tar.gz: 95750fac853da2683a7c41b452518bb027ca2696552fc7312b8075377c6ac47668fc55242488d52ce26328d2e96f70f4140fe7aeb31ea4adc98aba5abc68a4c1
+  metadata.gz: 2380c1c3b218440779ac943690c669464f8cebf0d774d78737044e29cd6ef61b980a9e790e414b47ed6761a99d273d26be8068af13c4dad1c3c4ac277daa1cb5
+  data.tar.gz: 29fb85145eef1e720c10da2a0edeb5d196f4e44f8629b14a6ca55e3c7d31e01a7af351094da9d88a180f5186b39fd6779809f746c816621c6d3629b45f13d5ec

data/CHANGELOG.md

@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.2.0] - 2026-10-18
+
+Support for Jev 1.14's experimental image input.
+
+- `TypeSafe::Image` from bytes, a file path, or base64 text. Place images in state, instructions or
+  criteria; they are uploaded as multipart assets. Images written into text go in `assets:`.
+- `box` questions and `TypeSafe::Responses::BoxAnswer` with `box` coordinates (`response.boxes`).
+- `danger_zone:` option on the client and per call to turn on `:images` and `:long_context`.
+- `TypeSafe::PayloadTooLargeError` (413) and `TypeSafe::UnsupportedMediaTypeError` (415).
+
 ## [0.1.0] - 2026-09-19
 
 Initial release.
@@ -17,5 +27,6 @@ Initial release.
 - Retries with backoff and `Retry-After`, keep-alive connections, logging and instrumentation hooks.
 - RBS signatures.
 
-[Unreleased]: https://github.com/hnegishi/typesafe-ai-ruby/compare/v0.1.0...HEAD
+[Unreleased]: https://github.com/hnegishi/typesafe-ai-ruby/compare/v0.2.0...HEAD
+[0.2.0]: https://github.com/hnegishi/typesafe-ai-ruby/compare/v0.1.0...v0.2.0
 [0.1.0]: https://github.com/hnegishi/typesafe-ai-ruby/releases/tag/v0.1.0

data/README.md

@@ -27,25 +27,33 @@ gem install typesafe-ai-ruby
 
 ## Usage
 
-Get an API key from the [TypeSafe console](https://console.typesafe.ai/settings/keys) and set it as `TYPESAFE_API_KEY`, or pass it to the client directly.
+Get an API key from the [TypeSafe console](https://console.typesafe.ai/keys) and export it as `TYPESAFE_API_KEY`:
+
+```sh
+export TYPESAFE_API_KEY=...
+```
+
+The client reads it automatically, so `TypeSafe::Client.new` needs no arguments. You can also pass `api_key:` explicitly.
 
 ```ruby
 require "typesafe-ai-ruby"
 
-client = TypeSafe::Client.new(api_key: ENV["TYPESAFE_API_KEY"])
+client = TypeSafe::Client.new
 
 response = client.system_one(
   state: "I was charged twice. Please fix this ASAP.",
   questions: {
-    department: TypeSafe::Choice.new(
+    department: {
+      type: :choice,
       instructions: "Which team should handle this?",
       criteria: { billing: "Payment issues", technical: "Bugs or integrations", sales: "Pricing questions" }
-    ),
-    frustration: TypeSafe::Score.new(
+    },
+    frustration: {
+      type: :score,
       instructions: "How frustrated is the customer?",
       criteria: ["Calm", "Frustrated but civil", "Very angry"]
-    ),
-    is_urgent: TypeSafe::Noul.new(instructions: "Does the message convey urgency?")
+    },
+    is_urgent: { type: :noul, instructions: "Does the message convey urgency?" }
   }
 )
 
@@ -58,21 +66,22 @@ response.model                            # => "jev-1.13.0"
 
 ### Questions
 
-There are three question types. `instructions` and every description can be a string, a Hash, or an Array when a sentence is not enough.
+Questions are plain Hashes matching the [API reference](https://docs.typesafe.ai/api): a `type`, optional `instructions`, and type-specific `criteria`. Instructions and descriptions can be a string, a Hash, or an Array.
+
+- `noul` asks a yes/no question and returns the probability of yes. Criteria are optional: `criteria: { true: "Spam", false: "A real conversation" }`.
+- `choice` picks one option from a set. Criteria are required; use `nil` when the name speaks for itself.
+- `score` rates the state against ordered levels. Criteria are an Array of at least two levels, and each level's index is its score.
 
-- `TypeSafe::Noul` asks a yes/no question and returns the probability of yes. Criteria are optional: `criteria: { true: "Spam", false: "A real conversation" }`.
-- `TypeSafe::Choice` picks one option from a set. Criteria are required; use `nil` when the name speaks for itself.
-- `TypeSafe::Score` rates the state against ordered levels. Criteria are an Array of at least two levels, and each level's index is its score.
+Extra keys are passed through untouched, so new API fields work before this library knows about them.
 
-The shorthand helpers take the instructions first:
+To reuse a question, keep a `TypeSafe::Noul`, `TypeSafe::Choice` or `TypeSafe::Score` in a constant and pass it in place of a Hash:
 
 ```ruby
-TypeSafe.noul("Is this spam?")
-TypeSafe.choice("What is the tone?", calm: nil, frustrated: nil, angry: nil)
-TypeSafe.score("How urgent is this?", ["Can wait", "This week", "Today"])
-```
+DEPARTMENT = TypeSafe::Choice.new(instructions: "Which team should handle this?",
+                                  criteria: { billing: nil, technical: nil, sales: nil })
 
-You can also pass a plain Hash with a `type` key. Extra keys are sent to the API untouched, so new API fields work before this library knows about them.
+client.system_one(state: ticket, questions: { department: DEPARTMENT })
+```
 
 `state` is whatever the questions are about: a String, a Hash, or an Array. Symbols are converted to strings, and objects that respond to `as_json` are converted through it.
 
@@ -110,13 +119,47 @@ end
 
 Malformed questions raise `TypeSafe::ValidationError` before anything is sent.
 
+### Images (experimental)
+
+Jev can look at PNG and JPEG images. The feature is behind a "danger zone" flag because its format and limits can still change, so turn it on per client or per call with `danger_zone: :images`. See the [images guide](https://docs.typesafe.ai/images) for limits and tips.
+
+Build a `TypeSafe::Image` from bytes, a file path, or base64 text, and place it anywhere text can go. It is uploaded once and referenced from the request:
+
+```ruby
+door = TypeSafe::Image.open("door-camera.jpg")
+
+response = client.system_one(
+  state: { door: door },
+  questions: {
+    person_present: { type: :noul, instructions: "Is there a person in `door`?" },
+    mug: { type: :box, instructions: "the coffee mug in `door`" }
+  },
+  model: "jev-preview",
+  danger_zone: :images
+)
+
+response.nouls[:person_present].noul   # => 0.97
+response.boxes[:mug].box.to_a          # => [left, top, right, bottom], as fractions of the image size
+```
+
+A `box` question returns where something is in an image and needs Jev 1.14 (`jev-preview`). To write an image into a sentence, interpolate it and pass it in `assets:`:
+
+```ruby
+client.system_one(
+  state: { photo: door },
+  questions: { same: TypeSafe.noul("Is the dog in `photo` the dog in #{poster}?") },
+  assets: [poster],
+  danger_zone: :images
+)
+```
+
 ## Configuration
 
 Options can be passed to `TypeSafe::Client.new` or set once for the default client. Explicit options win over environment variables, which win over the defaults.
 
 ```ruby
 TypeSafe.configure do |c|
-  c.api_key = ENV.fetch("TYPESAFE_API_KEY")   # TYPESAFE_API_KEY
+  c.api_key = ENV.fetch("TYPESAFE_API_KEY")
   c.model = "jev-1.13.0"                       # TYPESAFE_DEFAULT_MODEL, default jev-latest
   c.base_url = "https://api.typesafe.ai"       # TYPESAFE_BASE_URL
   c.timeout = 10                               # seconds per attempt
@@ -124,7 +167,7 @@ TypeSafe.configure do |c|
   c.log_level = :info                          # TYPESAFE_LOG_LEVEL, default :warn
 end
 
-TypeSafe.client.system_one(state: "...", questions: { urgent: TypeSafe.noul("Is this urgent?") })
+TypeSafe.client.system_one(state: "...", questions: { urgent: { type: :noul, instructions: "Is this urgent?" } })
 ```
 
 Per-call overrides go in `request_options`:

data/lib/typesafe/assets.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+module TypeSafe
+  # Turns the images in a request into the assets map sent with it.
+  #
+  # Images placed directly in state, instructions or criteria are collected while the request
+  # is normalized. Images written into text through their reference, or sent under a name of
+  # your choice, are passed in assets:. Every reference in the request must resolve to an
+  # image, and every attached image must be referenced, or the API rejects the request.
+  module Assets
+    MAX_REFERENCE_BYTES = 1024
+
+    module_function
+
+    # Normalize the assets: argument into reference => Image.
+    def attach(value)
+      case value
+      when nil then {}
+      when Image then { value.reference => value }
+      when Array then value.to_h { |image| [image!(image).reference, image] }
+      when Hash then value.to_h { |name, image| [reference!(name), image!(image)] }
+      else raise ValidationError, "assets must be an Image, an Array of images, or a Hash of reference => Image"
+      end
+    end
+
+    # Merge collected and attached images and check that references and assets match up.
+    # +json+ is the encoded request body without the assets map.
+    def resolve(collected, attached, json)
+      assets = collected.to_h { |image| [image.reference, image] }.merge(attached)
+      json.scan(Image::REFERENCE_PATTERN).uniq.each do |reference|
+        next if assets.key?(reference)
+
+        raise ValidationError, "#{reference} refers to an image that is not attached to the request; pass it in assets:"
+      end
+      attached.each_key do |reference|
+        next if json.include?(escape(reference))
+
+        raise ValidationError, "asset #{reference.inspect} is not referenced anywhere in the request"
+      end
+      assets.each_value(&:validate!)
+    end
+
+    def image!(value)
+      return value if value.is_a?(Image)
+
+      raise ValidationError, "assets must contain TypeSafe::Image values, got #{value.class}"
+    end
+
+    def reference!(name)
+      reference = name.to_s
+      raise ValidationError, "asset references must not be blank" if reference.strip.empty?
+      if reference.bytesize > MAX_REFERENCE_BYTES
+        raise ValidationError, "asset reference is #{reference.bytesize} bytes; the limit is #{MAX_REFERENCE_BYTES}"
+      end
+
+      reference
+    end
+
+    # A reference as it appears inside a JSON string.
+    def escape(reference)
+      JSON.generate(reference)[1...-1]
+    end
+  end
+end

data/lib/typesafe/client.rb

@@ -23,16 +23,26 @@ module TypeSafe
       @models = Resources::Models.new(@requestor)
     end
 
-    # Answer named questions about text or structured state.
-    def system_one(state:, questions:, model: nil, request_options: {})
-      options = RequestOptions.from(request_options)
+    # Answer named questions about text, structured state, or images.
+    #
+    # Images placed in state or questions are sent as assets automatically. Images written into
+    # text through their reference go in assets:, as an Array or as a Hash of your own
+    # reference names. Images and box questions need the :images danger zone turned on.
+    #
+    # danger_zone turns on experimental features for this call, such as :images; it is a
+    # shortcut for request_options: { danger_zone: [...] }.
+    def system_one(state:, questions:, model: nil, assets: nil, danger_zone: nil, request_options: {})
+      options = RequestOptions.from(request_options, danger_zone: danger_zone)
+      collected = []
       body = {
-        "state" => Questions::Normalizer.state(state),
+        "state" => Questions::Normalizer.state(state, assets: collected),
         "model" => resolve_model(model),
-        "questions" => Questions::Normalizer.questions(questions)
+        "questions" => Questions::Normalizer.questions(questions, assets: collected)
       }
       body.merge!(options.extra_body) if options.extra_body
-      response = @requestor.post(Constants::SYSTEM_ONE_PATH, body: body, options: options)
+      asset_map = Assets.resolve(collected, Assets.attach(assets), JSON.generate(body))
+      ensure_images_enabled!(asset_map, body["questions"], options)
+      response = @requestor.post(Constants::SYSTEM_ONE_PATH, body: body, options: options, assets: asset_map)
       Responses::SystemOneResponse.from_http(response)
     end
 
@@ -46,6 +56,23 @@ module TypeSafe
 
     private
 
+    def ensure_images_enabled!(assets, questions, options)
+      return if assets.empty? && questions.none? { |_, question| question["type"] == Questions::Box::TYPE }
+      return if images_enabled?(options)
+
+      raise ValidationError,
+            "images and box questions are experimental; turn them on with danger_zone: :images"
+    end
+
+    def images_enabled?(options)
+      return true if (config.danger_zone + options.danger_zone).include?(:images)
+
+      header = Constants::Headers::DANGER_ZONE_IMAGES.downcase
+      [config.headers, options.headers].any? do |headers|
+        headers.any? { |name, value| name.downcase == header && value.to_s.casecmp?("true") }
+      end
+    end
+
     def resolve_model(model)
       return config.model if model.nil?
       raise ValidationError, "model must be a non-blank String" if Util.blank?(model)

data/lib/typesafe/configuration.rb

@@ -28,9 +28,11 @@ module TypeSafe
     attr_accessor :retry_policy
     # Custom transport responding to #call(request); defaults to Net::HTTP.
     attr_accessor :transport
+    # Experimental features to turn on for every request: :images and/or :long_context.
+    attr_accessor :danger_zone
 
     def initialize(api_key: nil, base_url: nil, model: nil, timeout: nil, headers: nil, logger: nil,
-                   log_level: nil, retry_policy: nil, transport: nil)
+                   log_level: nil, retry_policy: nil, transport: nil, danger_zone: nil)
       @api_key = api_key
       @base_url = base_url
       @model = model
@@ -40,13 +42,15 @@ module TypeSafe
       @log_level = log_level
       @retry_policy = retry_policy
       @transport = transport
+      @danger_zone = danger_zone
     end
 
     # The explicitly set options, suitable for splatting into Client.new.
     def to_h
       {
         api_key: api_key, base_url: base_url, model: model, timeout: timeout, headers: headers,
-        logger: logger, log_level: log_level, retry_policy: retry_policy, transport: transport
+        logger: logger, log_level: log_level, retry_policy: retry_policy, transport: transport,
+        danger_zone: danger_zone
       }.compact
     end
 
@@ -59,6 +63,7 @@ module TypeSafe
       resolved.timeout = resolve_timeout
       resolved.headers = resolve_headers
       resolved.retry_policy = RetryPolicy.from(retry_policy)
+      resolved.danger_zone = resolve_danger_zone
       resolve_logging(resolved, env)
       resolved.freeze
     end
@@ -107,6 +112,12 @@ module TypeSafe
       (headers || {}).each_with_object({}) { |(name, value), result| result[name.to_s] = value.to_s }
     end
 
+    def resolve_danger_zone
+      Util.danger_zones(danger_zone).freeze
+    rescue ArgumentError => e
+      raise ConfigurationError, e.message
+    end
+
     def resolve_logging(resolved, env)
       resolved.log_level = resolve_log_level(env)
       resolved.logger = logger || build_logger(resolved.log_level)

data/lib/typesafe/constants.rb

@@ -44,6 +44,14 @@ module TypeSafe
       RUNTIME = "X-TypeSafe-Runtime"
       RETRY_COUNT = "X-TypeSafe-Retry-Count"
       REQUEST_ID = "x-typesafe-request-id"
+      DANGER_ZONE_IMAGES = "TypeSafe-Danger-Zone-Images"
+      DANGER_ZONE_LONG_CONTEXT = "TypeSafe-Danger-Zone-Long-Context"
     end
+
+    # Experimental features that must be turned on per request with a TypeSafe-Danger-Zone-* header.
+    DANGER_ZONES = {
+      images: Headers::DANGER_ZONE_IMAGES,
+      long_context: Headers::DANGER_ZONE_LONG_CONTEXT
+    }.freeze
   end
 end

data/lib/typesafe/errors.rb

@@ -65,6 +65,10 @@ module TypeSafe
   class PermissionDeniedError < APIError; end
   # The resource was not found (404).
   class NotFoundError < APIError; end
+  # The request body or an image is over a size limit (413).
+  class PayloadTooLargeError < APIError; end
+  # The request used an unsupported Content-Encoding (415).
+  class UnsupportedMediaTypeError < APIError; end
   # The request failed server-side validation (422).
   class UnprocessableEntityError < APIError; end
   # The server failed to process the request (5xx).
@@ -96,6 +100,8 @@ module TypeSafe
     401 => AuthenticationError,
     403 => PermissionDeniedError,
     404 => NotFoundError,
+    413 => PayloadTooLargeError,
+    415 => UnsupportedMediaTypeError,
     422 => UnprocessableEntityError,
     429 => RateLimitError,
     529 => OverloadedError

data/lib/typesafe/http/multipart_encoder.rb

@@ -0,0 +1,65 @@
+# frozen_string_literal: true
+
+require "securerandom"
+
+module TypeSafe
+  module HTTP
+    # Encodes a multipart/form-data body: the JSON request as the first part, named
+    # "request", followed by one file part per image asset.
+    class MultipartEncoder
+      Part = Struct.new(:name, :content_type, :content, keyword_init: true)
+      Encoded = Struct.new(:body, :content_type, :summary, keyword_init: true)
+
+      # Encode a request body Hash with its assets (reference => Image). The assets map in the
+      # JSON points each reference at its file part, named asset-0, asset-1 and so on.
+      def self.build(body, assets)
+        parts = parts_for(assets)
+        entries = assets.keys.zip(parts).to_h { |ref, part| [ref, { "type" => "image", "part" => part.name }] }
+        json = JSON.generate(body.merge("assets" => entries))
+        encoder = new
+        Encoded.new(body: encoder.encode(json, parts), content_type: encoder.content_type,
+                    summary: summary_for(json, parts))
+      end
+
+      def self.parts_for(assets)
+        assets.each_value.with_index.map do |image, index|
+          Part.new(name: "asset-#{index}", content_type: image.content_type, content: image.bytes)
+        end
+      end
+
+      def self.summary_for(json, parts)
+        "#{json} + #{parts.map { |p| "#{p.name} (#{p.content_type}, #{p.content.bytesize} bytes)" }.join(", ")}"
+      end
+
+      attr_reader :boundary
+
+      def initialize(boundary: SecureRandom.hex(16))
+        @boundary = boundary
+      end
+
+      def content_type
+        "multipart/form-data; boundary=#{boundary}"
+      end
+
+      # The encoded body as a binary String.
+      def encode(json, parts)
+        body = +"".b
+        append(body, "request", "application/json", json)
+        parts.each { |part| append(body, part.name, part.content_type, part.content, filename: part.name) }
+        body << "--#{boundary}--\r\n"
+      end
+
+      private
+
+      def append(body, name, content_type, content, filename: nil)
+        disposition = %(form-data; name="#{name}")
+        disposition += %(; filename="#{filename}") if filename
+        body << "--#{boundary}\r\n"
+        body << "Content-Disposition: #{disposition}\r\n"
+        body << "Content-Type: #{content_type}\r\n\r\n"
+        body << content.b
+        body << "\r\n"
+      end
+    end
+  end
+end

data/lib/typesafe/http/request.rb

@@ -5,13 +5,16 @@ module TypeSafe
     # An outgoing HTTP request: method, absolute URI, headers, encoded body and per-attempt timeout.
     class Request
       attr_reader :method, :uri, :headers, :body, :timeout
+      # What to show for the body in debug logs; multipart bodies summarize their file parts.
+      attr_reader :body_summary
 
-      def initialize(method:, uri:, headers:, body:, timeout:)
+      def initialize(method:, uri:, headers:, body:, timeout:, body_summary: nil)
         @method = method
         @uri = uri
         @headers = headers.freeze
         @body = body
         @timeout = timeout
+        @body_summary = body_summary || body
         freeze
       end
 
@@ -23,7 +26,8 @@ module TypeSafe
 
       # A copy with additional headers merged in.
       def with_headers(extra)
-        self.class.new(method: method, uri: uri, headers: headers.merge(extra), body: body, timeout: timeout)
+        self.class.new(method: method, uri: uri, headers: headers.merge(extra), body: body, timeout: timeout,
+                       body_summary: body_summary)
       end
     end
   end

data/lib/typesafe/http/requestor.rb

@@ -20,9 +20,10 @@ module TypeSafe
         @clock = clock
       end
 
-      # Send a POST and return the successful response.
-      def post(path, body:, options: RequestOptions.new)
-        execute(build_request(:post, path, body: body, options: options), options)
+      # Send a POST and return the successful response. With assets (reference => Image), the
+      # body goes out as multipart/form-data with one file part per image.
+      def post(path, body:, options: RequestOptions.new, assets: {})
+        execute(build_request(:post, path, body: body, options: options, assets: assets), options)
       end
 
       # Send a GET and return the successful response.
@@ -97,28 +98,43 @@ module TypeSafe
         RetryPolicy.from(options.retry_policy, base: config.retry_policy)
       end
 
-      def build_request(method, path, body: nil, options: RequestOptions.new)
-        encoded = body.nil? ? nil : JSON.generate(body)
+      def build_request(method, path, body: nil, options: RequestOptions.new, assets: {})
+        encoded, content_type, summary = encode_body(body, assets)
         Request.new(
           method: method,
           uri: URI.parse("#{config.base_url}#{path}"),
-          headers: build_headers(options.headers, body: encoded),
+          headers: build_headers(options.headers, content_type: content_type, danger_zone: options.danger_zone),
           body: encoded,
+          body_summary: summary,
           timeout: options.timeout || config.timeout
         )
       end
 
-      def build_headers(extra, body:)
+      # [body, content type, log summary] for a JSON body, or a multipart body when assets are present.
+      def encode_body(body, assets)
+        return [nil, nil, nil] if body.nil?
+        return [JSON.generate(body), Constants::JSON_CONTENT_TYPE, nil] if assets.empty?
+
+        encoded = MultipartEncoder.build(body, assets)
+        [encoded.body, encoded.content_type, encoded.summary]
+      end
+
+      def build_headers(extra, content_type:, danger_zone: [])
         headers = strip_protected(config.headers).merge(strip_protected(extra))
+        headers.merge!(danger_zone_headers(danger_zone))
         headers[Constants::Headers::AUTHORIZATION] = "Bearer #{config.api_key}"
         headers[Constants::Headers::ACCEPT] = Constants::JSON_CONTENT_TYPE
         headers[Constants::Headers::USER_AGENT] = sdk_identifier
         headers[Constants::Headers::SDK] = sdk_identifier
         headers[Constants::Headers::RUNTIME] = Util.runtime
-        headers[Constants::Headers::CONTENT_TYPE] = Constants::JSON_CONTENT_TYPE if body
+        headers[Constants::Headers::CONTENT_TYPE] = content_type if content_type
         headers
       end
 
+      def danger_zone_headers(zones)
+        (config.danger_zone + zones).uniq.to_h { |zone| [Constants::DANGER_ZONES.fetch(zone), "true"] }
+      end
+
       def strip_protected(headers)
         headers.reject { |name, _| PROTECTED_HEADERS.include?(name.downcase) }
       end
@@ -129,7 +145,8 @@ module TypeSafe
 
       def log_request(request)
         Logging.debug(config) do
-          "-> #{request.endpoint} headers={#{Logging.format_headers(request.headers)}} body=#{request.body.inspect}"
+          "-> #{request.endpoint} headers={#{Logging.format_headers(request.headers)}} " \
+            "body=#{request.body_summary.inspect}"
         end
       end
 

data/lib/typesafe/image.rb

@@ -0,0 +1,123 @@
+# frozen_string_literal: true
+
+require "securerandom"
+
+module TypeSafe
+  # A PNG or JPEG image to send with a request. Images are an experimental API feature and
+  # need the :images danger zone turned on.
+  #
+  # Build one from file bytes, a path (read when the request is sent), or base64 text, then
+  # place it anywhere text can go in state, instructions or criteria. Each image is sent once
+  # under its reference, and to_s returns that reference so an image can be written into a
+  # sentence; images used that way must be passed in assets: as well.
+  #
+  # For example:
+  #   door = TypeSafe::Image.open("door-camera.jpg")
+  #   client.system_one(
+  #     state: { door: door },
+  #     questions: { person: { type: :noul, instructions: "Is there a person in `door`?" } },
+  #     danger_zone: :images
+  #   )
+  class Image
+    MAX_BYTES = 16 * 1024 * 1024
+    REFERENCE_PATTERN = /<image:[0-9a-f]{32}>/
+    SIGNATURES = {
+      "image/png" => "\x89PNG\r\n\x1a\n".b,
+      "image/jpeg" => "\xFF\xD8\xFF".b
+    }.freeze
+
+    # The unique string this image is sent under, such as "<image:3f2a...>".
+    attr_reader :reference
+    # The path the image is read from, when built with Image.open.
+    attr_reader :path
+
+    # An image read from a file when the request is sent.
+    def self.open(path)
+      new(nil, path: path.to_s)
+    end
+
+    # An image from base64 text, such as an upload. Whitespace is ignored; a data: prefix is not allowed.
+    def self.from_base64(text)
+      raise ValidationError, "base64 image data must be a String, got #{text.class}" unless text.is_a?(String)
+
+      compact = text.gsub(/\s+/, "")
+      bytes = compact.unpack1("m0") # strict base64, the standard alphabet with padding
+      new(bytes)
+    rescue ArgumentError
+      raise ValidationError, "base64 image data is not valid base64"
+    end
+
+    # Whether a string is an image reference.
+    def self.reference?(string)
+      string.is_a?(String) && REFERENCE_PATTERN.match?(string) && string.match(REFERENCE_PATTERN)[0] == string
+    end
+
+    def initialize(bytes = nil, path: nil)
+      raise ArgumentError, "pass either image bytes or a path" if bytes.nil? == path.nil?
+
+      @path = path
+      @bytes = bytes.nil? ? nil : validate_bytes(bytes)
+      @reference = "<image:#{SecureRandom.hex(16)}>"
+    end
+
+    # The image file bytes, read from the path on first use.
+    def bytes
+      @bytes ||= validate_bytes(read_file)
+    end
+
+    def bytesize
+      bytes.bytesize
+    end
+
+    # "image/png" or "image/jpeg", from the file signature.
+    def content_type
+      SIGNATURES.find { |_, signature| bytes.start_with?(signature) }&.first ||
+        raise(ValidationError, "#{describe} is not a PNG or JPEG file")
+    end
+
+    # "png" or "jpeg".
+    def format
+      content_type.delete_prefix("image/")
+    end
+
+    # Check that the bytes are a PNG or JPEG within the size limit; raises ValidationError otherwise.
+    def validate!
+      content_type
+      self
+    end
+
+    def to_s
+      reference
+    end
+
+    def to_json(*args)
+      reference.to_json(*args)
+    end
+
+    def inspect
+      "#<#{self.class.name} #{path ? path.inspect : "#{@bytes&.bytesize || "?"} bytes"} #{reference}>"
+    end
+
+    private
+
+    def read_file
+      File.binread(path)
+    rescue SystemCallError => e
+      raise ValidationError, "cannot read image #{path.inspect}: #{e.message}"
+    end
+
+    def validate_bytes(data)
+      raise ValidationError, "image data must be a String of bytes, got #{data.class}" unless data.is_a?(String)
+      raise ValidationError, "#{describe} is empty" if data.empty?
+      if data.bytesize > MAX_BYTES
+        raise ValidationError, "#{describe} is #{data.bytesize} bytes; the limit is #{MAX_BYTES}"
+      end
+
+      data.b.freeze
+    end
+
+    def describe
+      path ? "image #{path.inspect}" : "image data"
+    end
+  end
+end