prompt_builder 0.1.0 → 0.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3b3f2f981ac5ffaacac8d0f46d3cde5b2429042f34846b36b6a0154fee9858f4
-  data.tar.gz: 7aceebc0337cd9f6f201ccbf822caca50d53ff23e3d2ed165bd7d114c9d9120f
+  metadata.gz: e762383ea51918fa77b5962f0137775551a37da1c228a37d24364011c8d21f73
+  data.tar.gz: 64e2a4e22feb0a8e0e5752f4807b79452c3cb700dd8b5740500f11cbe0b4217e
 SHA512:
-  metadata.gz: 0fdf71137c15a30e7e2cc5740bfa5fc4b7b9a2d31b9959821f84e879b718150dbfd054e15b19e13f53caead242d70ddaaf06e315e79044f77dce1500c6e374cf
-  data.tar.gz: 935a2ac05da1aa179f6107d46e0d37b80b79762111f95f16cda73924e9d9a48733cff1761413b4a8dedb56b8f880b517ed2cd9e196f5ea648e5ae13d909c9e10
+  metadata.gz: b7eb3a4806f9cd7a24e00652659b7a5d641ae5877187bd714afd0e4eaa86c7a7901eebcfdf11202497d0b98cce03f9a0fec10f1840883f91b4b87be3a9f2c387
+  data.tar.gz: 2b731def4d25b2233b208d3fb3d8867ee17cce9356cb84b3d57d3d3043116af5eb55ca7888de00c329c3637b209bbc70ea11dada040bc5f97f9701c85a0fa8ee

data/CHANGELOG.md

@@ -4,20 +4,29 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## Unreleased
+## 0.1.2
+
+### Fixed
+
+- `Session#add_function_call_output` raised a `NameError`; it now correctly builds an `Items::FunctionCallOutput`.
+- Provider-specific `extra` data on items, content blocks, and tool definitions (e.g. `cache_control`, `cache_point`, `thought_signature`, `media_type`) leaked into Open Responses request payloads and would be rejected as unknown parameters. It is now stripped from `request_payload(:open_responses)`.
+- The Messages serializer emitted all `OutputText.annotations` as text-block `citations`, including annotation shapes from other providers (e.g. Chat Completions `url_citation`) that the Anthropic API rejects. Only valid Anthropic citation types are now emitted.
+- The Chat Completions serializer emitted tool messages with an empty content array when a tool result contained no text content; it now collapses to an empty string.
+- The Chat Completions response parser duplicated message-level `annotations` and `logprobs` onto every text block; they are now attached only to the first.
+- `PromptBuilder.parse_data_url` now accepts data URLs with media type parameters (e.g. `data:text/plain;charset=utf-8;base64,...`).
+
+### Added
+
+- `Response#provider_data` as an alias for `Response#extra`.
+
+## 0.1.1
 
 ### Changed
 
-- Serializers now silently omit unsupported features from the serialized output
-  instead of raising `UnsupportedFormatError`. Unsupported session fields, content
-  types, enum values, sub-keys, `tool_choice` variants, and `Compaction`/`ItemReference`
-  items are dropped from request payloads; unrecognized blocks, candidates, and choices
-  are skipped when parsing responses. `UnsupportedFormatError` is still raised for
-  missing required data (e.g. no `model`, no messages, a `json_schema` format without a
-  schema), structural problems, genuine conflicts, unparseable function-call arguments,
-  and streaming response chunks.
+- Responses will now raise an `UnsupportedFormatError` if the response shape is missing key elements.
+- Removed the default `max_tokens` value of 4096 from the request serializer. If `max_output_tokens` is not set on the session, the request will simply omit the `max_tokens` parameter, allowing the API to apply its own defaults.
 
-## 1.0.0
+## 0.1.0
 
 ### Added
 

data/README.md

@@ -78,11 +78,11 @@ Once you've built a session, serialize it to a request payload for the API you w
 **Open Responses API**:
 
 ```ruby
-payload = session.to_h
-# or
 payload = session.request_payload(:open_responses)
 ```
 
+Note that `session.to_h` is *not* a request payload — it is the persistence format used by `Session.from_h` and keeps provider-specific `extra` data, canonical `url` content keys, and non-replayable items that the Open Responses API would reject. Always build request payloads through `request_payload`.
+
 **OpenAI Chat Completions API**:
 
 ```ruby
@@ -117,7 +117,7 @@ uri = URI("https://api.openai.com/v1/responses")
 request = Net::HTTP::Post.new(uri)
 request["Authorization"] = "Bearer #{api_key}"
 request["Content-Type"] = "application/json"
-request.body = JSON.generate(session.to_h)
+request.body = JSON.generate(session.request_payload(:open_responses))
 
 response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
   http.request(request)
@@ -535,7 +535,7 @@ The following table shows which session configuration fields are supported by ea
 ⁹ `RefusalContent` blocks are dropped silently by all request serializers so a parsed refusal can sit in session history without breaking subsequent `request_payload` calls. A message left empty after stripping is omitted entirely.
 ¹⁰ Chat Completions sends `InputFile` as a `{"type": "file", "file": {...}}` content block. `file_id` (Files API) and base64 data URL (with optional `filename`/`media_type`) are supported. A URL-only `InputFile` (non-data URL) is omitted because Chat Completions has no remote-URL form for files.
 ¹¹ `OutputText.annotations` (e.g. URL citations from `web_search_options`) are parsed onto the assistant message and round-trip through session history, but are dropped silently on Chat Completions and Converse request serialization since those formats have no request-side equivalent. `OutputText.logprobs` is likewise dropped.
-¹² Messages text-block `citations` are parsed into `OutputText.annotations` and emitted back as `citations` when serializing Messages history. Document/tool-result citation opt-ins are still not modeled.
+¹² Messages text-block `citations` are parsed into `OutputText.annotations` and emitted back as `citations` when serializing Messages history. Annotations from other providers (e.g. a Chat Completions `url_citation`) are silently dropped since they are not valid Anthropic citation shapes. Document/tool-result citation opt-ins are still not modeled.
 ¹³ Converse `requestMetadata` only accepts string key/value pairs. This serializer stringifies scalar metadata values and silently omits nested arrays or objects.
 
 ### Features Not Accessible Through Open Responses
@@ -617,7 +617,7 @@ Request-side mappings worth calling out:
 | `InputFile` with data URL (+ optional `filename`/`media_type`) | `{"type": "file", "file": {"filename": ..., "file_data": "data:<media_type>;base64,..."}}` |
 | `InputImage` with data URL (+ `media_type`) | `{"type": "image_url", "image_url": {"url": "data:<media_type>;base64,..."}}` |
 
-Unsupported Chat Completions request features not exposed by this gem include audio input/output (`audio`, `modalities`, `input_audio` content), `web_search_options`, custom tools, `tool_choice: {"type": "allowed_tools", ...}`, `seed`, `stop`, `logit_bias`, `n`, `prediction`, and the deprecated `functions`, `function_call`, `max_tokens`, and `user` fields. Use the modern canonical fields when available (`tools`, `tool_choice`, `max_output_tokens`, `safety_identifier`, `prompt_cache_key`).
+Chat Completions request features with no canonical Open Responses field are available through `session.extra` (`stop`, `seed`, `logit_bias`, `n`, `prediction`, `web_search_options`, `modalities`, `audio`). Features not exposed at all include `input_audio` content blocks, custom tools, `tool_choice: {"type": "allowed_tools", ...}`, and the deprecated `functions`, `function_call`, `max_tokens`, and `user` fields. Use the modern canonical fields when available (`tools`, `tool_choice`, `max_output_tokens`, `safety_identifier`, `prompt_cache_key`).
 
 Response-side behavior and limitations:
 
@@ -676,7 +676,7 @@ Content and message restrictions:
 - Assistant messages map to `role: "model"`; consecutive same-role turns are merged automatically.
 - `InputImage`, `InputFile`, and `InputVideo` are only supported in user messages; the same content on an assistant message is omitted.
 - `InputImage` accepts any URL, a base64 data URL (with required `media_type`), or a Files API `file_id`.
-- `InputFile` accepts any URL, a base64 data URL, or `file_id`. `media_type` is required when not inferable from a `filename` or URL extension. Recognized extensions: `pdf`, `txt`, `md`/`markdown`, `html`/`htm`, `csv`, `json`, `xml`, `rtf`.
+- `InputFile` accepts any URL, a base64 data URL, or `file_id`. `media_type` is required when not inferable from a `filename` or URL extension. Recognized extensions cover documents (`pdf`, `txt`, `md`/`markdown`, `html`/`htm`, `csv`, `json`, `xml`, `rtf`), images (`png`, `jpg`/`jpeg`, `webp`, `heic`, `heif`), audio (`mp3`, `wav`, `aiff`, `aac`, `ogg`, `flac`), and video (`mp4`, `mov`, `webm`, `mpeg`/`mpg`).
 - `InputVideo` requires a URL; raw bytes are not modeled.
 - `RefusalContent` is dropped silently so a parsed Chat Completions refusal can sit in session history without breaking subsequent serialization.
 - `Reasoning` items round-trip via `parts[].thought` with `thoughtSignature` preserved. `redacted_thinking`, `summary`, and unknown reasoning block types are silently skipped.
@@ -689,7 +689,7 @@ Response-side limitations:
 
 - Unknown response `Part` shapes (`inlineData`, `fileData`, `executableCode`, `codeExecutionResult`, `videoMetadata`, server-side `toolCall`/`toolResponse`, or any `Part` without a recognized content key) are silently skipped.
 - Only `candidates[0]` is parsed. When `candidateCount > 1`, additional candidates are dropped (their index is preserved on `provider_data`).
-- Function-call `id` from the response is dropped; the parser synthesizes `gemini_call_<seed>_<n>` so multiple calls in one response share a deterministic seed.
+- Function-call `id` from the response is preserved when present; otherwise the parser synthesizes `gemini_call_<seed>_<n>` so multiple calls in one response share a deterministic seed.
 - `finishReason` mappings: `STOP` → `completed`; `MAX_TOKENS` → `incomplete`; `SAFETY`, `RECITATION`, `OTHER`, `BLOCKLIST`, `PROHIBITED_CONTENT`, `SPII`, `MALFORMED_FUNCTION_CALL`, `IMAGE_SAFETY`, `LANGUAGE`, `UNEXPECTED_TOOL_CALL`, `TOO_MANY_TOOL_CALLS`, `MODEL_ARMOR` → `failed`. `FINISH_REASON_UNSPECIFIED` is treated as nil.
 - An empty `candidates` array combined with `promptFeedback.blockReason` is mapped to `failed`.
 - `usageMetadata.cachedContentTokenCount`, `toolUsePromptTokenCount`, `promptTokensDetails`, `cacheTokensDetails`, and `toolUsePromptTokensDetails` populate `response.usage.input_tokens_details`. `thoughtsTokenCount` and `candidatesTokensDetails` populate `response.usage.output_tokens_details`.
@@ -725,14 +725,15 @@ Content and message restrictions:
 - `FunctionCall.arguments` must parse to a JSON object; non-object JSON values raise (Bedrock's `toolUse.input` requires an object).
 - `FunctionCallOutput.output` content supports `InputText`/`OutputText`, `InputImage`, `InputFile`, and `InputVideo`, mapping to Converse `text`, `image`, `document`, and `video` tool result blocks. Converse also supports `json` and `searchResult` tool result blocks, but this gem has no canonical content types for them, so unsupported content is omitted. `FunctionCallOutput.status` is mapped: `completed` → `success`, `failed`/`incomplete` → `error`, anything else passes through or is dropped.
 - `tool_choice: "none"` and `tool_choice` without registered tools are both omitted.
-- Converse API request features not exposed by this gem include `stopSequences`, `additionalModelRequestFields`, `additionalModelResponseFieldPaths`, `guardrailConfig` / `guardContent`, `cachePoint`, Prompt Management `promptVariables`, audio blocks, `searchResult` blocks, and `performanceConfig.latency`.
+- Converse API request features with no canonical Open Responses field are available through `session.extra` (`stop_sequences`, `guardrail_config`, `additional_model_request_fields`, `additional_model_response_field_paths`, `performance_config`, `prompt_variables`) and content `extra` (`cache_point`). Features not exposed at all include `guardContent`, audio blocks, and `searchResult` blocks.
+- The serialized payload includes `modelId` as a convenience; the Converse REST endpoint takes the model id in the URL path (`/model/{modelId}/converse`), and AWS REST-JSON services ignore unrecognized body members. Remove it from the body if you prefer a strictly minimal payload.
 
 Response-side limitations:
 
 - Unknown content block keys (e.g. `citationsContent`, `guardContent`) are silently skipped.
 - `metrics.latencyMs`, `trace` (guardrail and prompt-router trace events), `additionalModelResponseFields`, `performanceConfig`, and the raw `serviceTier` object are exposed on `response.provider_data`. `response.service_tier` is also populated from `serviceTier.type`.
-- `stopReason` mappings: `end_turn` / `tool_use` / `stop_sequence` → `completed`, `max_tokens` / `model_context_window_exceeded` → `incomplete`, `guardrail_intervened` / `content_filtered` / `malformed_model_output` / `malformed_tool_use` → `failed`. Unlike the Messages serializer, the matched stop sequence text is not surfaced separately because Converse does not echo it back unless you request provider-specific fields through `additionalModelResponseFieldPaths`, which this gem cannot emit.
-- `usage.cacheReadInputTokens` and `usage.cacheWriteInputTokens` populate `response.usage.input_tokens_details["cached_tokens"]` and `["cache_creation_input_tokens"]`. Cache writes still require `cachePoint` markers in the request, which this gem cannot produce.
+- `stopReason` mappings: `end_turn` / `tool_use` / `stop_sequence` → `completed`, `max_tokens` / `model_context_window_exceeded` → `incomplete`, `guardrail_intervened` / `content_filtered` / `malformed_model_output` / `malformed_tool_use` → `failed`. Unlike the Messages serializer, the matched stop sequence text is not surfaced separately because Converse does not echo it back unless you request provider-specific fields through `additionalModelResponseFieldPaths` (available via the `additional_model_response_field_paths` session extra; the requested fields appear on `response.provider_data`).
+- `usage.cacheReadInputTokens` and `usage.cacheWriteInputTokens` populate `response.usage.input_tokens_details["cached_tokens"]` and `["cache_creation_input_tokens"]`. Cache writes require `cachePoint` markers in the request, which are emitted via the `cache_point` content extra.
 
 ## Installation
 

data/VERSION

@@ -1 +1 @@
-0.1.0
+0.1.2

data/lib/prompt_builder/errors.rb

@@ -15,4 +15,7 @@ module PromptBuilder
 
   # Raised when an operation is invalid for the current session state.
   class InvalidStateError < Error; end
+
+  # Raised when a response payload does not match the expected shape.
+  class UnexpectedPayloadError < Error; end
 end

data/lib/prompt_builder/response.rb

@@ -125,6 +125,14 @@ module PromptBuilder
     FIELDS.each { |f| attr_reader f }
     attr_reader :text_config, :output, :usage, :extra
 
+    # @!method provider_data
+    #   Provider-specific response metadata with no canonical Open Responses
+    #   slot (e.g. Gemini grounding metadata, Chat Completions system_fingerprint).
+    #   Alias for +extra+.
+    #
+    #   @return [Hash, nil]
+    alias_method :provider_data, :extra
+
     BOOLEAN_FIELDS.each { |f| alias_method("#{f}?", f) }
 
     # Create a new Response.

data/lib/prompt_builder/serializers/base.rb

@@ -31,6 +31,22 @@ module PromptBuilder
         def deserialize_response(_hash)
           raise NotImplementedError
         end
+
+        # Ensure a response payload is a Hash containing the key that identifies
+        # it as a well-formed response for this format. Raised before parsing so
+        # malformed bodies (e.g. provider error envelopes) fail loudly with the
+        # offending body rather than producing a silently empty Response.
+        #
+        # @param hash [Object] the raw response payload
+        # @param key [String] a key that must be present in a valid payload
+        # @raise [UnexpectedPayloadError] if the key is missing
+        def require_response_key!(hash, key)
+          return if hash.is_a?(Hash) && hash.key?(key)
+
+          body = JSON.generate(hash)[0..200]
+          raise UnexpectedPayloadError,
+            "unexpected response payload, missing #{key.inspect}: #{body}"
+        end
       end
     end
   end

data/lib/prompt_builder/serializers/chat_completion/request.rb

@@ -228,12 +228,16 @@ module PromptBuilder
 
             # Only text content is supported in tool output; other content types
             # are silently omitted.
-            output.filter_map do |content|
+            content = output.filter_map do |content|
               case content
               when Content::InputText, Content::OutputText
                 serialize_text_content(content)
               end
             end
+
+            # Chat Completions rejects tool messages with an empty content
+            # array; collapse to an empty string like the other serializers.
+            content.empty? ? "" : content
           end
 
           def flush_tool_calls!(messages, pending_tool_calls, last_assistant_msg = nil)

data/lib/prompt_builder/serializers/chat_completion/response.rb

@@ -36,6 +36,10 @@ module PromptBuilder
                   content: [Content::RefusalContent.new(refusal: message["refusal"])]
                 )
               elsif message["content"].is_a?(Array)
+                # logprobs and annotations are reported once per message, so
+                # attach them to the first text block only to avoid duplicating
+                # them across blocks on round-trip.
+                first_text_block = true
                 contents = message["content"].each_with_object([]) do |block, acc|
                   case block["type"]
                   when "text", nil
@@ -44,9 +48,10 @@ module PromptBuilder
 
                     acc << Content::OutputText.new(
                       text: text,
-                      logprobs: logprobs_content,
-                      annotations: annotations
+                      logprobs: first_text_block ? logprobs_content : [],
+                      annotations: first_text_block ? annotations : []
                     )
+                    first_text_block = false
                   when "refusal"
                     acc << Content::RefusalContent.new(refusal: block["refusal"]) if block["refusal"]
                   else
@@ -109,6 +114,8 @@ module PromptBuilder
             if hash["object"] == "chat.completion.chunk"
               raise UnsupportedFormatError, "Chat Completions streaming chunks are not supported"
             end
+            require_response_key!(hash, "choices")
+            require_response_key!(hash, "model")
             # Responses with multiple choices have no canonical multi-candidate
             # representation; only the first choice is parsed (handled by the
             # caller using choices[0]).

data/lib/prompt_builder/serializers/converse/request.rb

@@ -55,15 +55,17 @@ module PromptBuilder
       # === Features in Converse not available through Open Responses
       #
       # The following Converse parameters cannot be set through the Open Responses
-      # canonical format:
-      # - +stopSequences+ — custom stop sequences
-      # - Guardrail policies (+guardrailConfig+)
-      # - Per-model passthrough fields (+additionalModelRequestFields+)
-      # - Requested provider response fields (+additionalModelResponseFieldPaths+)
-      # - Cross-region routing via inference profiles
-      # - +performanceConfig+ latency settings beyond +serviceTier+
-      # - Prompt management variables (+promptVariables+)
-      # - Prompt caching markers (+cachePoint+)
+      # canonical format but are recognized via session +extra+ keys:
+      # - +stopSequences+ — custom stop sequences (+stop_sequences+)
+      # - Guardrail policies (+guardrail_config+)
+      # - Per-model passthrough fields (+additional_model_request_fields+)
+      # - Requested provider response fields (+additional_model_response_field_paths+)
+      # - +performanceConfig+ latency settings (+performance_config+)
+      # - Prompt management variables (+prompt_variables+)
+      #
+      # Prompt caching markers (+cachePoint+) are emitted via the +cache_point+
+      # content extra. Cross-region routing via inference profiles is selected
+      # through the model id and needs no request field.
       class Request < Base
         IMAGE_MEDIA_TYPE_FORMATS = {
           "image/jpeg" => "jpeg",

data/lib/prompt_builder/serializers/converse/response.rb

@@ -11,6 +11,9 @@ module PromptBuilder
           private
 
           def deserialize_response(hash)
+            require_response_key!(hash, "output")
+            require_response_key!(hash, "stopReason")
+
             usage_hash = hash["usage"]
             usage = if usage_hash
               cache_read = usage_hash["cacheReadInputTokens"]

data/lib/prompt_builder/serializers/gemini/response.rb

@@ -64,6 +64,9 @@ module PromptBuilder
           private
 
           def deserialize_response(hash)
+            require_response_key!(hash, "candidates")
+            require_response_key!(hash, "modelVersion")
+
             usage = build_usage(hash["usageMetadata"])
 
             # Only the first candidate is parsed; additional candidates have no

data/lib/prompt_builder/serializers/messages/request.rb

@@ -71,12 +71,22 @@ module PromptBuilder
       # - +anthropic-beta+ headers and API versioning (this gem produces no HTTP
       #   request — set headers in your HTTP client)
       class Request < Base
-        DEFAULT_MAX_TOKENS = 4096
         SUPPORTED_METADATA_KEYS = ["user_id"].freeze
         EFFORT_LEVELS = ["low", "medium", "high", "xhigh", "max"].freeze
         SUPPORTED_THINKING_TYPES = ["adaptive", "disabled", "enabled"].freeze
         SUPPORTED_TOOL_CHOICE_TYPES = ["any", "auto", "none", "tool"].freeze
 
+        # Citation types accepted by the Messages API on text blocks. Annotations
+        # parsed from other providers (e.g. Chat Completions url_citation) have
+        # no Anthropic equivalent and are silently dropped.
+        SUPPORTED_CITATION_TYPES = [
+          "char_location",
+          "content_block_location",
+          "page_location",
+          "search_result_location",
+          "web_search_result_location"
+        ].freeze
+
         class << self
           private
 
@@ -85,7 +95,7 @@ module PromptBuilder
             raise UnsupportedFormatError, "Messages format requires session.model" unless session.model
 
             h["model"] = session.model
-            h["max_tokens"] = session.max_output_tokens || DEFAULT_MAX_TOKENS
+            h["max_tokens"] = session.max_output_tokens if session.max_output_tokens
             h["temperature"] = session.temperature if session.temperature
             h["top_p"] = session.top_p if session.top_p
             effective_metadata = build_effective_metadata(session)
@@ -341,7 +351,8 @@ module PromptBuilder
               {"type" => "text", "text" => content.text}
             when Content::OutputText
               text = {"type" => "text", "text" => content.text}
-              text["citations"] = content.annotations unless content.annotations.empty?
+              citations = serialize_citations(content.annotations)
+              text["citations"] = citations unless citations.empty?
               text
             when Content::InputImage
               # Assistant image content is not supported; omit it.
@@ -425,6 +436,16 @@ module PromptBuilder
             end
           end
 
+          # Only annotations that are valid Anthropic citation objects can be
+          # replayed as citations. Annotations from other providers (e.g. a
+          # Chat Completions url_citation) are silently dropped so a
+          # cross-provider session history doesn't produce an invalid request.
+          def serialize_citations(annotations)
+            annotations.select do |annotation|
+              annotation.is_a?(Hash) && SUPPORTED_CITATION_TYPES.include?(annotation["type"])
+            end
+          end
+
           # FunctionCallOutput.status values that map to tool_result.is_error.
           # "incomplete" and "failed" cover the OR canonical statuses; "error"
           # is accepted as a convenience alias.

data/lib/prompt_builder/serializers/messages/response.rb

@@ -11,6 +11,9 @@ module PromptBuilder
           private
 
           def deserialize_response(hash)
+            require_response_key!(hash, "content")
+            require_response_key!(hash, "stop_reason")
+
             usage_hash = hash["usage"]
             usage = build_usage(usage_hash) if usage_hash
 

data/lib/prompt_builder/serializers/open_responses/request.rb

@@ -5,6 +5,15 @@ module PromptBuilder
     class OpenResponses < Base
       # Request serializer for the OpenAI Open Responses API format.
       class Request < Base
+        # Content extra keys that are part of the Open Responses API schema and
+        # must survive extra-stripping (the canonical content classes have no
+        # dedicated attribute for them).
+        CANONICAL_EXTRA_KEYS = {
+          "input_file" => ["file_id"].freeze,
+          "input_image" => ["file_id"].freeze
+        }.freeze
+        private_constant :CANONICAL_EXTRA_KEYS
+
         class << self
           # Export a session to Open Responses API request payload.
           #
@@ -12,9 +21,9 @@ module PromptBuilder
           # @return [Hash] the serialized request payload
           def request_payload(session)
             payload = session.to_h
-            apply_server_state!(payload, session)
+            items = apply_server_state!(payload, session)
             payload.delete("extra")
-            strip_extra(payload)
+            strip_extra(payload, items, session.tool_definitions)
             normalize_content_urls!(payload)
             strip_non_replayable_reasoning!(payload)
             strip_output_only_fields!(payload)
@@ -28,36 +37,78 @@ module PromptBuilder
           # When the session is in server-state mode, replace the full input
           # array with only items added after the last response boundary and
           # include previous_response_id for the API to resolve history.
+          # Returns the item objects backing the payload's input array.
           def apply_server_state!(payload, session)
-            return if session.local_state?
+            return session.items if session.local_state?
 
             payload.delete("input")
-            new_items = session.items[session.response_boundary_index..]
-            payload["input"] = new_items.map(&:to_h) if new_items && !new_items.empty?
+            new_items = session.items[session.response_boundary_index..] || []
+            payload["input"] = new_items.map(&:to_h) unless new_items.empty?
+            new_items
           end
 
-          # Walk the payload and remove any "extra" keys from items, content
-          # blocks, and tool definitions since they are not part of the Open
-          # Responses API schema.
-          def strip_extra(payload)
+          # Remove provider-specific extra data from items, content blocks, and
+          # tool definitions since it is not part of the Open Responses API
+          # schema. Items, content blocks, and tool definitions serialize their
+          # extras flat into the hash, so the extra keys must be looked up on
+          # the backing objects rather than removed as a nested "extra" key.
+          def strip_extra(payload, items, tool_definitions)
             input = payload["input"]
             if input.is_a?(Array)
-              input.each do |item|
-                next unless item.is_a?(Hash)
+              input.each_with_index do |item_hash, index|
+                next unless item_hash.is_a?(Hash)
+
+                item = items[index]
+                next unless item
 
-                item.delete("extra")
+                strip_extra_keys!(item_hash, item.extra)
 
-                if item["type"] == "function_call_output" && item["output"].is_a?(Array)
-                  strip_extra_from_blocks!(item["output"])
-                else
-                  content = item["content"]
-                  strip_extra_from_blocks!(content) if content.is_a?(Array)
+                case item
+                when Items::Message
+                  strip_extra_from_blocks!(item_hash["content"], item.content)
+                when Items::FunctionCallOutput
+                  strip_extra_from_blocks!(item_hash["output"], item.output) if item.output.is_a?(Array)
                 end
               end
             end
 
             tools = payload["tools"]
-            strip_extra_from_blocks!(tools) if tools.is_a?(Array)
+            if tools.is_a?(Array)
+              tools.each_with_index do |tool_hash, index|
+                next unless tool_hash.is_a?(Hash)
+
+                definition = tool_definitions[index]
+                strip_extra_keys!(tool_hash, definition.extra) if definition
+              end
+            end
+          end
+
+          def strip_extra_from_blocks!(blocks, contents)
+            return unless blocks.is_a?(Array)
+
+            blocks.each_with_index do |block, index|
+              next unless block.is_a?(Hash)
+
+              content = contents[index]
+              next unless content.respond_to?(:extra)
+
+              keep = CANONICAL_EXTRA_KEYS.fetch(block["type"], [])
+              strip_extra_keys!(block, content.extra, keep: keep)
+            end
+          end
+
+          def strip_extra_keys!(hash, extra, keep: [])
+            return unless extra.is_a?(Hash)
+
+            extra.each_key do |key|
+              # "type" is canonical on every item and content block; an extra
+              # named "type" never survives serialization, so deleting it here
+              # would corrupt the block.
+              next if key == "type"
+              next if keep.include?(key)
+
+              hash.delete(key)
+            end
           end
 
           # Convert canonical "url" keys back to Open Responses API keys:
@@ -201,14 +252,6 @@ module PromptBuilder
             normalized.empty? ? nil : normalized
           end
 
-          def strip_extra_from_blocks!(blocks)
-            blocks.each do |block|
-              next unless block.is_a?(Hash)
-
-              block.delete("extra")
-            end
-          end
-
           # Normalize text.format for the Responses API. If the format uses
           # the Chat Completions nested json_schema sub-object, flatten it
           # so name/schema/strict/description sit directly under format.

data/lib/prompt_builder/serializers/open_responses/response.rb

@@ -9,6 +9,9 @@ module PromptBuilder
           private
 
           def deserialize_response(hash)
+            require_response_key!(hash, "status")
+            require_response_key!(hash, "object")
+
             PromptBuilder::Response.from_h(hash)
           end
         end

data/lib/prompt_builder/session.rb

@@ -210,10 +210,10 @@ module PromptBuilder
     # Add a tool call output to the conversation.
     #
     # @param call_id [String] the tool call identifier
-    # @param result [String, Hash, Array, Content::Base, nil] the tool call result
-    # @return [Items::FunctionOutput] the added function output item
+    # @param result [String, Array<Content::Base, Hash>, nil] the tool call result
+    # @return [Items::FunctionCallOutput] the added function call output item
     def add_function_call_output(call_id:, result:)
-      add_item(Items::FunctionOutput.new(call_id: call_id, result: result))
+      add_item(Items::FunctionCallOutput.new(call_id: call_id, output: result))
     end
 
     # Add a raw item to the conversation.

data/lib/prompt_builder.rb

@@ -44,14 +44,16 @@ module PromptBuilder
       "data:#{content_type};base64,#{[data].pack("m0")}"
     end
 
-    # Parse a data URL into its media type and base64-encoded data.
+    # Parse a data URL into its media type and base64-encoded data. Media type
+    # parameters (e.g. "data:text/plain;charset=utf-8;base64,...") are allowed
+    # and excluded from the returned media type.
     #
     # @param url [String, nil] a URL that may be a data URL
     # @return [Array(String, String), nil] a two-element array of +[media_type, data]+
-    #   or nil if the URL is not a data URL
+    #   or nil if the URL is not a base64-encoded data URL
     def parse_data_url(url)
       return nil unless url
-      match = url.match(/\Adata:([^;]+);base64,(.*)\z/m)
+      match = url.match(/\Adata:([^;,]+)(?:;[^,]*)?;base64,(.*)\z/m)
       return nil unless match
       [match[1], match[2]]
     end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: prompt_builder
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.2
 platform: ruby
 authors:
 - Brian Durand