turnkit 0.7.0 → 0.7.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2a276982d34a5c7685f9512e29f1843942166c67bdb59c94ebc38b727213ffe3
-  data.tar.gz: 8ad4edfdc514c24a918441734c4c10294e7d1c45b08b7594f19db7e3294edd61
+  metadata.gz: fb1f58cfea02169c68e433d8a8b84dc5eb2a8a54f18ea9e89853e4ad85445ccd
+  data.tar.gz: '085c56655a5e11670528b157926a6003a04a074944adaf7cf61730cc18c5d6a3'
 SHA512:
-  metadata.gz: 5bff210317826da7bd796b0271006b76c2ed301dbd9070248529efd8c14e3b2d43414258373f6c709a1496f91004764227ff2453073be0040ac0d5bde4eb3d77
-  data.tar.gz: ed8c88ae93a81983f2284e3ed69c018a1e05abd4b2ae081db2160a937c1e4e42fdece546baac10ca8c976e94da54354275f15d80d5dc4cc60cb55e919a78e8d1
+  metadata.gz: 38ef2c605e8ee0d5a15417db75a6efce91b045103f68f86a241f7db788806b63ab6dbdd16cc40b1bee24b54ed93f6b83fb9331e13dbf6d06900bc79f4e7305d8
+  data.tar.gz: a9a752449f11422b599b9eaba4598441a295366fc4ab1dddccb136689d9279c2e8b426de9d1f2f8b45fe74011209bd22942e388033892547b5f8355a65d8e56c

data/CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## 0.7.2 - 2026-09-10
+
+- Add explicit `Tool.budget_completion!` for replay-safe local terminal saves
+  from an already-billed response that reaches the spend limit. Persist the
+  exact eligible call ID with the response; skip other batch calls and preserve
+  ordinary authorization, validation, receipts, fencing and recovery. Ambiguous
+  or failed saves fail without another model request.
+- Treat exact spend exhaustion as a bound and recheck persisted spend before
+  model, media and ordinary tool dispatch. Terminal status alone does not permit
+  over-budget execution; the opt-in does not waive other runtime limits.
+
+## 0.7.1 - 2026-09-10
+
+- Preserve OpenAI prompt prefixes with durable, append-only dynamic context
+  snapshots instead of recombining changing data into leading instructions.
+  Deduplicate unchanged snapshots across retries/resume, preserve opaque
+  Responses reasoning and tool pairing, and refresh full context after
+  compaction. Other clients retain the separate-instructions contract.
+- Add the `dynamic_context` message kind (no schema migration; upgrade workers
+  and custom kind allowlists together). Exclude snapshots from public progress
+  reads. Clarify that `prompt_cache: :off` does not disable OpenAI implicit caching.
+
 ## 0.7.0 - 2026-09-10
 
 - Add destination-oriented `Conversation#post`, durable input/request receipts,

data/README.md

@@ -323,6 +323,50 @@ class SaveBrief < TurnKit::Tool
 end
 ```
 
+### Saving an already-billed final response at the spend limit
+
+By default, reaching `max_spend` stops the turn before further work. A model
+response can itself reach or exceed that limit: its usage and proposed tool
+calls are persisted, but an ordinary terminal tool is not allowed to run.
+
+For a **local, idempotent final save only**, explicitly opt in on the tool:
+
+```ruby
+class SavePacket < TurnKit::Tool
+  terminal! { |result| "Saved #{result.fetch('id')}." }
+  recovery :replay_safe
+  budget_completion!
+
+  # Define parameters and call normally. Validate the complete packet before
+  # saving; persist context.idempotency_key with the save in one transaction.
+end
+```
+
+`budget_completion!` requires both terminal behavior and `recovery :replay_safe`.
+It is an application promise that the tool only validates/saves already acquired
+output locally: no model calls, acquisition, or child launches. Terminal status
+alone never grants this exception, and the marker is not inherited implicitly.
+
+When the billed response reaches the spend limit, TurnKit atomically records the
+sole eligible call ID with that response. The normal tool runner executes only
+that call; other calls in the batch get skipped receipts and paired tool results,
+even if they precede the save. Zero or multiple eligible calls fail without tool
+dispatch. No new model request is allowed in the exhausted turn, including
+compaction, model-backed output audits, image generation, or media analysis.
+
+Authorization, argument validation, claim fencing, cancellation, timeout/depth,
+and global/per-tool execution limits still apply. Invalid saves must raise
+`ToolValidationError`/`ToolError`; an ordinary error-shaped hash is still tool
+result data. Failed receipts are never replayed or repaired with another model
+call. Failed local output audits also terminate rather than request revision;
+model-backed audits cannot run at exhaustion. Use local validation for this path.
+
+Worker recovery reuses the persisted call/execution and its idempotency key;
+completed receipts are not re-executed. The application must make the save and
+its receipt atomic. An already-started external effect cannot be recalled by
+cancellation, so this marker must not be applied to acquisition tools. No schema
+migration is needed; upgrade workers together before enabling the opt-in.
+
 ### Output audits and policies
 
 Use output audits for deterministic checks that should not depend on another
@@ -463,6 +507,48 @@ agent = TurnKit::Agent.new(
 `TurnKit.prompt_behavior`, and `TurnKit.context_contributors` remain available
 for generated prompts.
 
+### Dynamic context and OpenAI prompt caching
+
+Generated prompts keep stable instructions separate from subject, live context,
+and environment. With the RubyLLM OpenAI adapter (Responses or Chat Completions),
+TurnKit persists each changed **full context snapshot** as a `dynamic_context`
+conversation message before model dispatch. It replays older snapshots unchanged
+and appends new ones after completed tool exchanges. They render as labeled user
+reference-data messages, not new user requests or top-level instructions;
+the newest snapshot replaces earlier snapshots for current state. Keep policies
+in stable instructions and supply current data through context contributors.
+
+Identical snapshots are deduplicated against durable, model-visible history,
+including after retry/resume. Empty context clears a previous snapshot. Changed
+context adds history, so keep contributors concise; compaction can remove old
+snapshots and resets that portion of the prefix. Always supply the full current
+context, not deltas. Changing tools, skills, schemas, model settings, or stable
+instructions can also invalidate a prefix. Existing histories are not rewritten.
+
+Custom clients retain the separate `instructions` / `dynamic_instructions`
+contract. Clients opting into `dynamic_context_in_history?(model:)` receive
+snapshots in `messages` and empty `dynamic_instructions`. Other clients do not
+receive these historical snapshot messages. Raw `Conversation#messages` includes
+them; public `messages_after` progress reads exclude them. No schema migration is
+needed, but custom message-kind allowlists must accept `dynamic_context`, and
+workers reading those conversations must be upgraded together.
+
+Direct `adapter.chat` callers still receive fresh dynamic context at the tail,
+but must preserve prior snapshots themselves (using
+`TurnKit::MessageProjection.dynamic_context(text)` in their own history) and
+avoid supplying the same snapshot again through `dynamic_instructions`.
+A custom `system_prompt:` string/callable is treated entirely as stable
+instructions; recombining `prompt.dynamic` there bypasses this protection.
+
+`TurnKit.prompt_cache = :auto` enables TurnKit's existing Anthropic cache markers;
+`:off` suppresses those markers. It is **not an OpenAI cache-disable switch**.
+OpenAI implicit caching remains provider-default in either setting, including
+when RubyLLM uses `store: false` or a fresh chat object. TurnKit does not force
+explicit breakpoints, cache keys, or TTLs. Matching wire prefixes make reuse
+possible, not guaranteed: eligibility, routing, lifetime, and model-specific
+boundaries still matter. See [OpenAI's prompt-caching guide](https://developers.openai.com/api/docs/guides/prompt-caching)
+and measure actual cache reads/writes before claiming savings.
+
 ### Tools
 
 Create a tool:
@@ -1016,7 +1102,9 @@ TurnKit.timeout = 300
 ```
 
 `max_spend` is the only spend-limit name in the public API.
-
+Spend equal to the limit is exhausted, not just spend above it. Dispatch checks
+use persisted aggregate spend, including after recovery and before internal
+model/media calls. Committed response cost is retained even when the turn fails.
 
 Customize cost rates with USD-per-million-token component keys:
 

data/lib/turnkit/adapters/ruby_llm.rb

@@ -30,6 +30,18 @@ module TurnKit
         raise ModelAccessError, "#{key_name} is required for #{model}. Set ENV[#{key_name.inspect}] or configure RubyLLM before running TurnKit."
       end
 
+      def dynamic_context_in_history?(model:)
+        ensure_ruby_llm!
+        # Resolve the catalog provider without creating a connection or
+        # requiring credentials, so prompt previews remain offline/read-only.
+        ::RubyLLM.models.find(model).provider.to_s == "openai"
+      rescue ConfigError
+        raise
+      rescue ::RubyLLM::ModelNotFoundError
+        # Preserve unknown-model previews; normal dispatch reports SDK errors.
+        false
+      end
+
       def chat(model:, messages:, tools:, instructions:, dynamic_instructions: nil, temperature: nil, thinking: nil, output_schema: nil, metadata: nil, on_event: nil)
         ensure_ruby_llm!
         configure_from_environment
@@ -37,7 +49,8 @@ module TurnKit
         validate!(model: model) if @protocol
         chat = ::RubyLLM.chat(**{ model: model, protocol: @protocol }.compact)
         chat.with_provider_options(metadata: metadata.transform_values(&:to_s)) if @protocol == :responses && metadata
-        add_instructions(chat, instructions, dynamic_instructions, model: model)
+        context_in_history = chat.model.provider.to_s == "openai"
+        add_instructions(chat, instructions, context_in_history ? nil : dynamic_instructions, model: model)
         chat.with_temperature(temperature) if temperature
         apply_thinking(chat, thinking)
         chat.with_schema(normalize_schema(output_schema)) if output_schema
@@ -46,6 +59,11 @@ module TurnKit
         end
         tool_names = {}
         Array(messages).each { |message| add_message(chat, message, provider: chat.model.provider, tool_names: tool_names) }
+        # The runtime supplies durable snapshots in messages. Direct adapter
+        # callers must preserve this message themselves on subsequent calls.
+        if context_in_history && !dynamic_instructions.to_s.empty?
+          add_message(chat, MessageProjection.dynamic_context(dynamic_instructions))
+        end
 
         response = complete_without_tool_execution(chat)
         normalize_response(response, model: model)

data/lib/turnkit/budget.rb

@@ -67,14 +67,18 @@ module TurnKit
 
       @mutex.synchronize do
         @cost += cost.to_f
-        raise BudgetError, "cost limit reached" if @cost > max_spend
+        raise BudgetError, "cost limit reached" if spend_exhausted?
       end
     end
 
-    def check!(depth:)
+    def spend_exhausted?
+      max_spend && @cost >= max_spend
+    end
+
+    def check!(depth:, allow_exhausted_spend: false)
       raise BudgetError, "maximum sub-agent depth reached" if max_depth && depth > max_depth
       raise BudgetError, "turn timed out" if timeout && Clock.now >= root_started_at + timeout
-      raise BudgetError, "cost limit reached" if max_spend && @cost > max_spend
+      raise BudgetError, "cost limit reached" if spend_exhausted? && !allow_exhausted_spend
     end
 
     private

data/lib/turnkit/client.rb

@@ -12,6 +12,13 @@ module TurnKit
       true
     end
 
+    # Opt in to durable, append-only dynamic context messages. The runtime
+    # then passes empty dynamic_instructions; ordinary clients keep the
+    # existing separate-instructions contract and do not see these messages.
+    def dynamic_context_in_history?(model:)
+      false
+    end
+
     def chat(model:, messages:, tools:, instructions:, dynamic_instructions: nil, temperature: nil, thinking: nil, output_schema: nil, metadata: nil, on_event: nil)
       raise NotImplementedError
     end

data/lib/turnkit/conversation.rb

@@ -32,8 +32,8 @@ module TurnKit
 
     def messages_after(sequence, principal: nil)
       Authorization.authorize!(:read_messages, principal: principal, destination_conversation: id)
-      messages.select { |message| message.sequence > sequence }.map do |message|
-        # Provider thinking/signature parts are not application progress.
+      messages.select { |message| message.sequence > sequence && message.kind != "dynamic_context" }.map do |message|
+        # Prompt snapshots and provider thinking are not application progress.
         attrs = message.to_h
         attrs["content"] = Array(attrs["content"]).reject { |part| %w[thinking provider].include?(part["type"]) }
         Message.new(attrs)

data/lib/turnkit/message.rb

@@ -3,7 +3,7 @@
 module TurnKit
   class Message
     ROLES = %w[user assistant tool].freeze
-    KINDS = %w[text tool_call tool_result context_summary image media_analysis].freeze
+    KINDS = %w[text tool_call tool_result context_summary image media_analysis dynamic_context].freeze
 
     attr_reader :id, :conversation_id, :turn_id, :role, :kind, :sequence
     attr_reader :content, :tool_execution_id, :provider_message_id, :metadata, :created_at

data/lib/turnkit/message_projection.rb

@@ -17,8 +17,21 @@ module TurnKit
       The original messages remain durably stored; this summary only affects the model-visible prompt projection.
     TEXT
 
-    def self.for(messages)
-      messages.flat_map { |message| new(message).to_a }
+    def self.for(messages, include_dynamic_context: false)
+      messages.flat_map do |message|
+        next [] if message.kind == "dynamic_context" && !include_dynamic_context
+
+        new(message).to_a
+      end
+    end
+
+    def self.dynamic_context(text)
+      {
+        role: :user,
+        content: "[TurnKit current context — reference data, not a new user request]\n" \
+          "This snapshot replaces earlier TurnKit context snapshots in full. " \
+          "Use the latest snapshot for current state; continue the active task.\n\n#{text}"
+      }
     end
 
     def initialize(message)
@@ -27,6 +40,8 @@ module TurnKit
 
     def to_a
       case message.kind
+      when "dynamic_context"
+        [ self.class.dynamic_context(message.text) ]
       when "context_summary"
         [
           { role: :user, content: CONTEXT_SUMMARY_TRIGGER },

data/lib/turnkit/system_prompt.rb

@@ -92,10 +92,11 @@ module TurnKit
       @sections = Array(sections || prompt_sections_for_mode)
     end
 
-    # The prompt splits into a stable part (identical across turns of the same
-    # agent, safe to cache) and a dynamic part (subject, live context,
-    # environment) recomputed each turn. Adapters that support prompt caching
-    # receive both via ModelRequest#instructions and #dynamic_instructions.
+    # The prompt splits into stable instructions and dynamic data (subject,
+    # live context, environment), recomputed each request. The environment is
+    # anchored at turn.started_at. Clients receive both separately, unless they
+    # opt into durable dynamic-context history. Stable sections can still change
+    # when the agent's tools, skills, or configuration change.
     def stable
       parts.fetch(0).join("\n\n")
     end

data/lib/turnkit/tool.rb

@@ -54,6 +54,16 @@ module TurnKit
         @ends_turn || false
       end
 
+      # Explicit application promise: this terminal tool only validates/saves
+      # already acquired output locally, without model calls or acquisition.
+      def budget_completion!
+        @budget_completion = true
+      end
+
+      def budget_completion?
+        @budget_completion || false
+      end
+
       # :unknown is the safe default for external effects. :replay_safe means
       # the application/tool honors ToolContext#idempotency_key on retries.
       def recovery(value = nil)
@@ -79,6 +89,9 @@ module TurnKit
       def validate_definition!
         raise ArgumentError, "tool name is required" if tool_name.empty?
         raise ArgumentError, "invalid tool name: #{tool_name}" unless NAME_PATTERN.match?(tool_name)
+        if budget_completion? && (!ends_turn? || recovery != :replay_safe)
+          raise ArgumentError, "budget_completion! requires a terminal tool with recovery :replay_safe"
+        end
 
         parameters.each do |param|
           type = param.fetch(:type)
@@ -188,6 +201,7 @@ module TurnKit
     def input_schema = self.class.input_schema
     def validate_definition! = self.class.validate_definition!
     def ends_turn? = self.class.ends_turn?
+    def budget_completion? = self.class.budget_completion?
     def completion_message(result) = self.class.completion_message(result)
   end
 end

data/lib/turnkit/tool_runner.rb

@@ -7,6 +7,20 @@ module TurnKit
     end
 
     def dispatch(tool_calls)
+      completion_id = turn.budget_completion_call_id
+      if completion_id
+        control = turn.control_boundary!
+        return control if control
+        selected = tool_calls.select { |call| call.id == completion_id }
+        tool = tool_for(selected.first&.name)
+        unless selected.length == 1 && tool&.budget_completion? && tool.ends_turn?
+          raise BudgetError, "budget completion is no longer available"
+        end
+        # Preserve pairing and receipts, regardless of where acquisition calls
+        # appear in the response. Only the durably selected call may execute.
+        skip_remaining(tool_calls - selected, terminal: selected.first)
+        tool_calls = selected
+      end
       waiting = false
       tool_calls.each_with_index do |tool_call, index|
         control = turn.control_boundary!
@@ -24,6 +38,7 @@ module TurnKit
           skip_remaining(tool_calls.drop(index + 1), terminal: tool_call)
           return execution
         end
+        raise BudgetError, "budget completion failed" if completion_id
       end
       waiting ? :waiting : nil
     end
@@ -82,6 +97,7 @@ module TurnKit
           # external-effect boundary. Calls already sent cannot be recalled.
           control = turn.control_boundary!
           return control if control
+          turn.execution_budget.check!(depth: turn.depth, allow_exhausted_spend: turn.budget_completion_call_id == tool_call.id)
           if turn.background? && subagent?(tool)
             return delegate(tool, tool_call, context)
           end
@@ -164,7 +180,8 @@ module TurnKit
         calls.each do |call|
           turn.store.atomic do
             next if turn.store.list_tool_executions(turn_id: turn.id).any? { |row| row["tool_call_id"] == call.id }
-            payload = { "skipped" => true, "message" => "not executed: turn ended by #{terminal.name}" }
+            reason = turn.budget_completion_call_id ? "spend limit reached" : "turn ended by #{terminal.name}"
+            payload = { "skipped" => true, "message" => "not executed: #{reason}" }
             execution = ToolExecution.new(create_execution(call))
             attrs = turn.store.claim_tool_execution(execution.id, from: execution.status, to: "cancelled", result: payload, completed_at: Clock.now)
             append_result_once(ToolExecution.new(attrs), call, payload)

data/lib/turnkit/turn.rb

@@ -125,7 +125,7 @@ module TurnKit
     end
 
     def preview
-      model_request
+      model_request(persist_context: false)
     end
 
     def status
@@ -151,6 +151,10 @@ module TurnKit
       options.dig("state", "policy_audit") || options["policy_audit"]
     end
 
+    def budget_completion_call_id
+      @record.dig("options", "state", "budget_completion_call_id")
+    end
+
     # Reads iterations from options["state"], falling back to the legacy
     # top-level key for turns persisted before the state split.
     def self.iterations_for(record)
@@ -188,6 +192,7 @@ module TurnKit
     end
 
     def internal_model_call(model:, messages:, instructions:, tools: [], thinking: nil, output_schema: nil, metadata: {}, purpose:, client: nil)
+      execution_budget.check!(depth: depth)
       request = ModelRequest.new(
         model: model,
         messages: messages,
@@ -303,8 +308,8 @@ module TurnKit
             break
           end
           @budget = execution_budget
-          budget.check!(depth: depth)
           state = @record.dig("options", "state") || {}
+          budget.check!(depth: depth, allow_exhausted_spend: budget_completion_call_id && %w[tools output].include?(state["phase"]))
           case state["phase"] || "model"
           when "model"
             count_iteration!
@@ -320,10 +325,11 @@ module TurnKit
               add_usage!(result.usage, cost: cost)
               persist_assistant_message(result)
               update_state!("phase" => result.tool_calls? ? "tools" : "output", "parts" => result.parts,
-                "candidate" => result.text, "output_data" => result.output_data, "terminal_tool_name" => nil)
+                "candidate" => result.text, "output_data" => result.output_data, "terminal_tool_name" => nil,
+                "budget_completion_call_id" => select_budget_completion(result))
             end
             emit_model_completed("model.completed", result, cost, model: model)
-            budget.add_cost!(cost.total)
+            budget.add_cost!(cost.total) unless budget_completion_call_id
           when "tools"
             runner = ToolRunner.new(self)
             terminal = runner.dispatch(Result.new(parts: state.fetch("parts")).tool_calls)
@@ -344,6 +350,9 @@ module TurnKit
           when "output"
             candidate = state.fetch("candidate")
             audit = check_policy(candidate, output_data: state["output_data"])
+            if budget_completion_call_id && audit && !audit.clean?
+              raise BudgetError, "budget completion rejected: #{audit.messages.join('; ')}"
+            end
             revisions_used = state["revisions_used"].to_i
             if should_revise?(audit, revisions_used)
               store.atomic do
@@ -380,7 +389,7 @@ module TurnKit
         heartbeat&.value
       end
 
-      def model_request
+      def model_request(persist_context: true)
         prompt = SystemPrompt.new(agent: agent, turn: self, conversation: conversation, mode: prompt_mode || agent.effective_prompt_mode(turn: self))
         instructions, dynamic_instructions = case agent.system_prompt
         when nil
@@ -390,9 +399,22 @@ module TurnKit
         else
           [ agent.system_prompt.call(prompt).to_s, nil ]
         end
+        client = agent.effective_client
+        context_in_history = client.respond_to?(:dynamic_context_in_history?) && client.dynamic_context_in_history?(model: model)
+        messages = llm_messages(include_dynamic_context: context_in_history)
+        if context_in_history
+          # Compare only model-visible snapshots: compaction can remove the
+          # previous one. Store before dispatch so worker recovery replays it.
+          previous = TurnKit::Compaction.project(conversation.messages_for_turn(self)).reverse.find { |message| message.kind == "dynamic_context" }
+          if previous ? previous.text != dynamic_instructions.to_s : !dynamic_instructions.to_s.empty?
+            conversation.append_message(role: "user", kind: "dynamic_context", text: dynamic_instructions.to_s, turn_id: id) if persist_context
+            messages << MessageProjection.dynamic_context(dynamic_instructions.to_s)
+          end
+          dynamic_instructions = nil
+        end
         ModelRequest.new(
           model: model,
-          messages: llm_messages,
+          messages: messages,
           tools: agent.effective_tools(turn: self),
           instructions: instructions,
           dynamic_instructions: dynamic_instructions,
@@ -405,6 +427,7 @@ module TurnKit
 
       # Clients implement the TurnKit::Client keyword contract. See client.rb.
       def call_client(request, client: agent.effective_client)
+        execution_budget.check!(depth: depth)
         client.chat(
           model: request.model,
           messages: request.messages,
@@ -419,14 +442,27 @@ module TurnKit
       end
 
       def call_image_client(client, request)
+        execution_budget.check!(depth: depth)
         with_heartbeat { client.paint(**request, on_event: ->(event) { emit_event(event) }) }
       end
 
       def call_media_client(client, request)
+        execution_budget.check!(depth: depth)
         with_heartbeat { client.view_media(**request, on_event: ->(event) { emit_event(event) }) }
       end
 
-      def llm_messages
+      def select_budget_completion(result)
+        return unless execution_budget.spend_exhausted?
+
+        tools = agent.effective_tools(turn: self)
+        candidates = result.tool_calls.select do |call|
+          tool = tools.find { |candidate| candidate.tool_name == call.name }
+          tool&.budget_completion? && tool.ends_turn?
+        end
+        candidates.first.id if candidates.length == 1
+      end
+
+      def llm_messages(include_dynamic_context: false)
         messages = TurnKit::Compaction.project(conversation.messages_for_turn(self))
         # Delivery time is not application time. A next-turn message can arrive
         # between an earlier turn's tool call/result or before its steering.
@@ -442,7 +478,7 @@ module TurnKit
           end
           receiver ? [receiver.fetch("context_message_sequence"), 1, message.sequence] : [message.sequence, 0, 0]
         end
-        MessageProjection.for(messages)
+        MessageProjection.for(messages, include_dynamic_context: include_dynamic_context)
       end
 
       def emit_model_requested(type, request)

data/lib/turnkit/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module TurnKit
-  VERSION = "0.7.0"
+  VERSION = "0.7.2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: turnkit
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.7.2
 platform: ruby
 authors:
 - Sam Couch