turnkit 0.4.1 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 67553a737fbce38e2402167aeb2cc799c55390c5c7c9740a8d73d056b0679a06
-  data.tar.gz: fc395c09f05e8ba640ec9dda4907419f0a76a47e8a45607435c5dde0630c3562
+  metadata.gz: 3db08f0653a468f22daa74070c7b1a290f02b44beaf6d35c950f3d36d86dc224
+  data.tar.gz: 18d3b2df593865ba53e0863d0e2b061f8744bdbbf78409b40d525c3ba47b5070
 SHA512:
-  metadata.gz: 45864840f7bc24d3626e0e3bb849f2ea3d71c1fdb8a2faa7ff19725ab0b6932d205962208c2ea75be7014d8f0befa7c6ebd754e0f5dfedd7db6875fff90254a9
-  data.tar.gz: 7dc83b0922078e52fb220438097514c0efba0d10740a9cec40e810f316d42c6e50fdc67f92db2cb9e9564fdcab376a1e01286d8821effa4a9246e51f8baf6c1c
+  metadata.gz: dd2216c9e2275dd737d6e1cf7dadd97053ca5afb6992939791669f29e047f3e44eb45f492a09b4494222117428baf07ffd9aa60352f9363b0a8b3eedb9c6ab54
+  data.tar.gz: 3b8f1a45bc98506dc09a9a6703aacf1f614fb7649c3c64942c98e902aff1db9232f986aad33c077be51fec1623d82abf4506d32698b34e3079ff775b9a1eb38b

data/CHANGELOG.md

@@ -1,5 +1,39 @@
 # Changelog
 
+## 0.5.0 - 2026-07-22
+
+### Breaking
+
+- Custom stores: `find_stale_turns` is replaced by `reconcile_stale_turns(before:)`, which must atomically transition eligible turns to `stale` and return the reconciled records. `update_tool_execution` is replaced by `claim_tool_execution(id, from:, to:, **attributes)`, an atomic compare-and-set mirroring `claim_turn`.
+
+### Fixed
+
+- Make stale-turn reconciliation atomic, so `TurnKit.reconcile_stale!` can no longer overwrite a turn that was concurrently claimed, heartbeated, or completed (#1).
+- Heartbeat while a tool executes, so tools slower than `TurnKit.timeout` are not falsely reconciled.
+
+### Added
+
+- Add an `interrupted` tool-execution status. Reconciliation marks a stale turn's unfinished tool executions `interrupted`, appends a synthetic error tool result for unresolved tool calls (keeping the transcript continuable), and emits `turn.stale` and `tool_call.interrupted` events. Late tool results arriving after interruption are dropped, never overwriting the reconciled state.
+- `TurnKit.reconcile_stale!` returns the reconciled turn records. `stale` is provisional: a worker whose heartbeats were merely late finishes its turn normally, replacing `stale` with the actual outcome.
+
+## 0.4.2 - 2026-07-02
+
+### Breaking
+
+- Remove `TurnKit::Workflow`; use `TurnKit::Agent.new(orchestrator: true, ...)` for reusable autonomous task runners.
+- Make `Agent#run` task positional, remove `TurnKit.model` aliases, and rename run accessors to `output_text` and `tool_executions`.
+- Simplify tool context injection to `context:`, reserve `context` as a tool parameter name, and reduce `SubAgentTool` input to `task`.
+- Replace prompt contributor APIs with per-agent `system_prompt:` customization and stable/dynamic `TurnKit::SystemPrompt` accessors.
+- Strictly validate custom cost-rate keys, remove `MediaAnalysisResult#structured?`, remove Rails record-class globals, and require atomic custom store `claim_turn` implementations.
+- Remove `ruby_llm` as a runtime dependency; add `gem "ruby_llm", "~> 1.16"` when using the default RubyLLM adapter.
+- Call custom clients with the full `TurnKit::Client` keyword contracts, including `dynamic_instructions:` for split stable/dynamic prompts.
+
+### Changed
+
+- Keep the Rails install generator under the conventional `lib/generators` path without a Railtie.
+- Document compaction config with symbol keys while continuing to accept string keys.
+- Persist turn runtime state under `options["state"]` with backward-compatible reads.
+
 ## 0.4.1 - 2026-06-19
 
 - Add first-class media analysis with `Turn#view_media`, `TurnKit.view_media`, and `TurnKit::ViewMediaTool`.

data/README.md

@@ -4,8 +4,8 @@
 [![Ruby](https://img.shields.io/badge/ruby-%3E%3D%203.1-red.svg)](https://www.ruby-lang.org)
 [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE.md)
 
-Build durable Ruby and Rails agents with conversations, runs, workflows, tools,
-skills, output audits, sub-agents, and persistence.
+Build durable Ruby and Rails agents with conversations, runs, orchestrator agents,
+tools, skills, output audits, sub-agents, and persistence.
 
 ## Installation
 
@@ -13,6 +13,9 @@ Add this line to your application's **Gemfile**:
 
 ```ruby
 gem "turnkit"
+
+# Required only when using TurnKit's default RubyLLM adapter.
+gem "ruby_llm", "~> 1.16"
 ```
 
 Run:
@@ -21,7 +24,7 @@ Run:
 bundle install
 ```
 
-Upgrading from an earlier TurnKit version? See the [Upgrade Guide](UPGRADE.md).
+Upgrading from an earlier TurnKit version? See the [0.4.2 Upgrade Guide](UPGRADE_TO_0_4_2.md).
 
 ## Quick Start
 
@@ -49,11 +52,11 @@ turn = agent.conversation.ask("Explain Ruby blocks in one sentence.")
 puts turn.output_text
 ```
 
-Or run a non-interactive application task:
+Or run a non-interactive application task.
 
 ```ruby
 run = agent.run("Explain Ruby blocks in one sentence.")
-puts run.output
+puts run.output_text
 ```
 
 ## Usage
@@ -63,9 +66,9 @@ For runnable, API-key-free examples of the three core entry points, see
 
 - conversation: durable thread over time;
 - agent run: one bounded application task;
-- workflow: reusable task runner with skills, tools, and limits.
+- orchestrator agent: reusable task runner with skills, tools, and limits.
 
-For fuller workflow examples, see:
+For fuller orchestrator examples, see:
 
 - [`examples/workflow_researcher`](examples/workflow_researcher): source-grounded research with web tools, batch reads, per-tool limits, and deep monitoring;
 - [`examples/amazon_memo_writer`](examples/amazon_memo_writer): strict memo generation with research tools, a structured terminal submit tool, deterministic format checks, and an LLM output policy.
@@ -75,14 +78,14 @@ For fuller workflow examples, see:
 Set a model:
 
 ```ruby
-TurnKit.model = "gpt-4.1-mini"
+TurnKit.default_model = "gpt-4.1-mini"
 ```
 
 Or configure TurnKit in one place:
 
 ```ruby
 TurnKit.configure do |config|
-  config.model = "gpt-4.1-mini"
+  config.default_model = "gpt-4.1-mini"
   config.max_spend = 0.25
   config.max_iterations = 12
 end
@@ -116,7 +119,7 @@ Then configure TurnKit:
 ```ruby
 TurnKit.configure do |config|
   config.client = TurnKit::Adapters::Codex.new(sandbox: "read-only")
-  config.model = "gpt-5.4"
+  config.default_model = "gpt-5.4"
 end
 ```
 
@@ -186,26 +189,26 @@ small wrapper over TurnKit's existing conversation and turn engine. Existing
 Prepare a pending run without calling the model:
 
 ```ruby
-run = agent.run(task: "Classify later.", async: true)
+run = agent.run("Classify later.", async: true)
 request = run.preview
 run.run!
 ```
 
-### Workflows
+### Orchestrator agents
 
-Use a workflow when a run graduates into a reusable production capability: a
-named task runner with workflow skills, tools, defaults, guardrails, compaction,
-and output policy.
+Use an orchestrator agent when a run graduates into a reusable production
+capability: a named task runner with skills, tools, defaults, guardrails,
+compaction, and output policy.
 
-Workflows fight for their life when the task has a repeatable operating
-procedure: inspect app data, gather context, use sources, draft, verify, save,
-and stop under budget. They are overkill for simple classification or extraction
-runs.
+Orchestrator agents fight for their life when the task has a repeatable
+operating procedure: inspect app data, gather context, use sources, draft,
+verify, save, and stop under budget. They are overkill for simple classification
+or extraction runs.
 
 ```ruby
 source_grounded_brief = TurnKit::Skill.from_file("app/ai/skills/source_grounded_brief.md")
 
-workflow = TurnKit::Workflow.new(
+agent = TurnKit::Agent.new(
   name: "brief_writer",
   instructions: "Create source-grounded briefs and verify claims before final output.",
   skills: [source_grounded_brief],
@@ -220,16 +223,17 @@ workflow = TurnKit::Workflow.new(
   compaction: {
     context_limit: 64_000,
     threshold: 0.75
-  }
+  },
+  orchestrator: true
 )
 
-run = workflow.run(
+run = agent.run(
   "Create a source-grounded brief.",
   input: { topic: "Rails 8 Solid Queue" }
 )
 
-puts run.output
-puts run.tool_calls.map(&:tool_name)
+puts run.output_text
+puts run.tool_executions.map(&:tool_name)
 puts run.cost.total
 ```
 
@@ -240,11 +244,11 @@ model-tool loop:
 model → tool → result → model → tool → result → final
 ```
 
-For repeated workflows, keep instructions, skills, and tools stable and pass the
+For repeated orchestrator runs, keep instructions, skills, and tools stable and pass the
 per-run data through `input:`. This gives provider prompt caching the best chance
-to reuse the stable workflow prompt while each run supplies dynamic data.
+to reuse the stable agent prompt while each run supplies dynamic data.
 
-### Choosing runs, conversations, and workflows
+### Choosing runs, conversations, and orchestrator agents
 
 Use the smallest entry point that matches the shape of work:
 
@@ -252,7 +256,7 @@ Use the smallest entry point that matches the shape of work:
 | --- | --- | --- |
 | `Conversation` | A user or app will keep adding messages over time. | Best for durable threads and follow-up steering; history grows, so long threads need compaction. |
 | `Agent#run` | Your app needs one bounded result now. | Best for simple production tasks; repeated complex policies can sprawl across callers. |
-| `TurnKit::Workflow` | A task becomes a named reusable workflow with tools, skills, limits, and observability. | Best cache and packaging story for repeated autonomous work; overkill for one-off/simple tasks. |
+| `Agent.new(orchestrator: true)` | A task becomes a named reusable agent with tools, skills, limits, and observability. | Best cache and packaging story for repeated autonomous work; overkill for one-off/simple tasks. |
 
 Prompt caching and compaction solve different problems:
 
@@ -262,7 +266,7 @@ Prompt caching and compaction solve different problems:
 - budgets (`max_spend`, `max_iterations`, `max_tool_executions`) keep autonomous
   loops bounded.
 
-Use `max_tool_executions_by_name` when a workflow needs different budgets for
+Use `max_tool_executions_by_name` when an orchestrator needs different budgets for
 different tools. For example, allow many cheap reads but only one final submit
 tool, or cap web searches while allowing a batch page reader.
 
@@ -270,19 +274,25 @@ Reach for separate agents and `sub_agents` only when the isolation is worth the
 extra model calls, such as different models, different tool permissions,
 parallel specialist review, or separate durable child conversations.
 
-Run a workflow with `run`:
+Run an orchestrator agent with `run`:
 
 ```ruby
-run = workflow.run(
-  "Create compliant outreach for this account.",
-  input: lead.attributes,
+outreach_agent = TurnKit::Agent.new(
+  name: "outreach_writer",
+  instructions: "Create compliant outreach for accounts.",
   max_spend: 0.25,
   max_iterations: 8,
   max_tool_executions: 20,
   compaction: {
     context_limit: 64_000,
     threshold: 0.75
-  }
+  },
+  orchestrator: true
+)
+
+run = outreach_agent.run(
+  "Create compliant outreach for this account.",
+  input: lead.attributes
 )
 ```
 
@@ -329,10 +339,11 @@ numbered_lists_only = ->(output) do
   }
 end
 
-workflow = TurnKit::Workflow.new(
+agent = TurnKit::Agent.new(
   name: "memo_writer",
   output_policy: [no_em_dash, numbered_lists_only],
-  output_policy_mode: :fail
+  output_policy_mode: :fail,
+  orchestrator: true
 )
 ```
 
@@ -354,18 +365,19 @@ policy can be a `.md`, `.markdown`, or `.txt` file path, a `TurnKit::Skill`, a
 `TurnKit::OutputPolicy`, or any object that responds to `#call` or `#check`.
 
 ```ruby
-workflow = TurnKit::Workflow.new(
+agent = TurnKit::Agent.new(
   name: "memo_writer",
   output_policy: "app/ai/policies/amazon_memo.md",
   output_policy_model: "gpt-4.1-mini",
   output_policy_thinking: { effort: :low },
-  output_policy_mode: :report
+  output_policy_mode: :report,
+  orchestrator: true
 )
 ```
 
 `output_policy_mode: :report` records violations while allowing the run to
 complete. `:fail` marks the run failed after recording the output and audit;
-`:fail` is the default for contract-driven workflows. Policy model usage and
+`:fail` is the default for contract-driven orchestrator runs. Policy model usage and
 cost are counted on the parent run.
 
 Add `output_retries:` to turn policy failures into bounded revision loops instead
@@ -374,7 +386,7 @@ of dead ends:
 ```ruby
 voice = TurnKit::Skill.from_file("app/ai/skills/memo_voice.md")
 
-workflow = TurnKit::Workflow.new(
+agent = TurnKit::Agent.new(
   name: "memo_writer",
   skills: [voice],
   output_policy: [voice, no_em_dash],
@@ -383,7 +395,8 @@ workflow = TurnKit::Workflow.new(
     "type" => "object",
     "required" => ["project_id"],
     "properties" => { "project_id" => { "type" => "string" } }
-  }
+  },
+  orchestrator: true
 )
 ```
 
@@ -418,6 +431,27 @@ Run the reviewed turn:
 turn.run!
 ```
 
+
+### Prompt customization
+
+Customize generated prompts with `system_prompt:` on an agent. A string replaces
+the generated prompt. A callable receives the built `TurnKit::SystemPrompt` and
+returns the final string:
+
+```ruby
+agent = TurnKit::Agent.new(
+  name: "reporter",
+  system_prompt: ->(prompt) {
+    [prompt.stable, prompt.section(:tools), prompt.dynamic].reject(&:empty?).join("\n\n")
+  }
+)
+```
+
+`TurnKit::SystemPrompt` supports `to_s`, `section(:tools)`, `stable`, and
+`dynamic`. `prompt_sections:`, `TurnKit.prompt_sections`,
+`TurnKit.prompt_behavior`, and `TurnKit.context_contributors` remain available
+for generated prompts.
+
 ### Tools
 
 Create a tool:
@@ -479,7 +513,7 @@ image.to_blob   # generated bytes for base64 responses, or fetched URL bytes
 image.mime_type # "image/png"
 ```
 
-For reusable workflow steps, subclass `TurnKit::ImageTool`:
+For reusable agent steps, subclass `TurnKit::ImageTool`:
 
 ```ruby
 class GenerateHeaderImage < TurnKit::ImageTool
@@ -546,7 +580,7 @@ media = TurnKit::MediaInput.bytes(
 )
 ```
 
-For reusable workflow steps, subclass `TurnKit::ViewMediaTool`:
+For reusable agent steps, subclass `TurnKit::ViewMediaTool`:
 
 ```ruby
 class ReviewHeaderImage < TurnKit::ViewMediaTool
@@ -717,12 +751,6 @@ Compact manually:
 conversation.compact!(focus: "billing migration")
 ```
 
-Run the local smoke test:
-
-```sh
-ruby script/manual_compaction.rb
-```
-
 ### Rails
 
 Install Rails persistence:
@@ -745,12 +773,37 @@ app/ai/tools/
 app/ai/skills/
 ```
 
-Reconcile stale turns:
+Use custom Active Record classes by passing class names to the store:
+
+```ruby
+TurnKit.store = TurnKit::ActiveRecordStore.new(
+  conversation_class: "My::Conversation",
+  turn_class: "My::Turn",
+  message_class: "My::Message",
+  tool_execution_class: "My::ToolExecution"
+)
+```
+
+Reconcile turns abandoned by a dead worker (for example after a hard-killed
+process). Run this periodically:
 
 ```ruby
 TurnKit.reconcile_stale!
 ```
 
+Reconciliation atomically marks pending and running turns whose last heartbeat
+is older than `TurnKit.timeout` as `stale`, so it never overwrites a turn that
+was concurrently claimed, heartbeated, or completed. Each stale turn's
+unfinished tool executions become `interrupted`, and a synthetic error tool
+result is appended for any unresolved tool call so the conversation can be
+continued. TurnKit never reruns an interrupted tool — whether its side effect
+happened is unknown, so the continued model is told not to assume either way.
+
+Reconciliation does not cancel or reassign the underlying process. If the
+original worker is still alive (its heartbeats were merely late), it continues
+over the synthetic interrupted result and later replaces `stale` with its
+actual `completed` or `failed` outcome — treat `turn.stale` as provisional.
+
 ## Options
 
 | Option | Description |
@@ -782,6 +835,25 @@ TurnKit.timeout = 300
 
 `max_spend` is the only spend-limit name in the public API.
 
+
+Customize cost rates with USD-per-million-token component keys:
+
+```ruby
+TurnKit.cost_rates["custom-model"] = {
+  input: 0.15,
+  output: 0.60,
+  cache_read: 0.03,
+  cache_write: 0.18,
+  thinking: 0.60
+}
+```
+
+Custom clients should subclass `TurnKit::Client` or accept the full `#chat`
+keyword contract, including `dynamic_instructions:`. Adapters that support prompt
+caching should cache `instructions` and append `dynamic_instructions` per turn.
+
+Custom stores must implement `claim_turn` atomically.
+
 Set options per agent:
 
 ```ruby
@@ -805,7 +877,10 @@ agent = TurnKit::Agent.new(
 
 ## Upgrading
 
-Add `output_data` for structured output persistence.
+See the [0.4.2 Upgrade Guide](UPGRADE_TO_0_4_2.md) for the full API migration checklist.
+
+Rails installs from older versions may need `output_data` for structured output,
+image, and media-analysis persistence.
 
 ```ruby
 add_column :turnkit_turns, :output_data, :json

data/UPGRADE.md

@@ -1,5 +1,34 @@
 # Upgrade Guide
 
+## 0.4.2
+
+TurnKit 0.4.2 is a pre-1.0 surface cleanup. Update these call sites before upgrading:
+
+- `TurnKit::Workflow.new(...)` → `TurnKit::Agent.new(orchestrator: true, ...)`. Orchestrator agents prepend the same autonomous-orchestrator preamble and default `prompt_mode` to `:task`.
+- `workflow.run(task, max_spend: ...)` / other per-run workflow overrides → configure those limits on the `Agent`; agents are cheap to construct.
+- `workflow.agent(**overrides)` → construct another `TurnKit::Agent` with the desired options.
+- `agent.run(task: "...")` → `agent.run("...", input: ..., async: ...)`.
+- `TurnKit.model` / `TurnKit.model=` → `TurnKit.default_model` / `TurnKit.default_model=`.
+- `run.output` → `run.output_text`.
+- `run.tool_calls` → `run.tool_executions`.
+- `run.steps` and `run.persisted?` were removed.
+- Tool framework context is always passed as `context:`. The `turnkit_context:` alternative was removed, and `context` is now a reserved tool parameter name.
+- `TurnKit::SubAgentTool` exposes a single model-visible `task` parameter.
+- Prompt extension hooks `TurnKit.system_prompt_contributors`, `TurnKit.model_prompt_contributors`, `TurnKit::PromptContribution`, and prompt section override objects were removed. Pass `system_prompt:` to `TurnKit::Agent` instead: a string replaces the generated prompt; a callable receives `TurnKit::SystemPrompt` and returns the final string.
+- `TurnKit::SystemPrompt` supports `to_s`, `section(:tools)`, `stable`, and `dynamic`. `TurnKit.prompt_sections`, per-agent `prompt_sections:`, `TurnKit.prompt_behavior`, and `TurnKit.context_contributors` remain.
+- `TurnKit.cost_rates` accepts only `input`, `output`, `cache_read`, `cache_write`, and `thinking` keys, expressed as USD per million tokens. Old aliases such as `cached_input`, `cache_creation`, `reasoning`, and `*_per_million` now raise `TurnKit::ConfigError`.
+- `MediaAnalysisResult#structured?` → `MediaAnalysisResult#data?`.
+- Rails record class globals (`TurnKit.conversation_record_class`, turn/message/tool_execution variants) were removed. Pass class names to the store: `TurnKit::ActiveRecordStore.new(conversation_class: "My::Conversation", turn_class: "My::Turn", message_class: "My::Message", tool_execution_class: "My::ToolExecution")`.
+- The Railtie was removed. `rails g turnkit:install` still works through the conventional `lib/generators` path.
+- `ruby_llm` is no longer a runtime dependency. To use the default RubyLLM adapter, add `gem "ruby_llm", "~> 1.16"` to your Gemfile. The Codex adapter and custom `TurnKit::Client` subclasses need no extra gems.
+- Custom clients are called with the full `TurnKit::Client` keyword contracts for `chat`, `paint`, and `view_media`; signature sniffing was removed. Subclass `TurnKit::Client` or accept the full keyword sets.
+- `Client#chat` gained `dynamic_instructions:`. Cache the stable `instructions` string and append `dynamic_instructions` per turn when the provider supports prompt caching.
+- `SystemPrompt::CACHE_BOUNDARY`, `split_cache_boundary`, and the `has_cache_boundary` prompt report field were removed. `SystemPrompt#report` keeps `stable_chars` and `dynamic_chars`.
+- `TurnKit::Store#claim_turn` no longer has a default implementation. Custom stores must implement an atomic compare-and-set claim.
+- Compaction configuration examples now use symbol keys; string keys are still accepted.
+
+Non-breaking: turn runtime state such as iterations and policy audit now persists under `options["state"]` with backward-compatible reads.
+
 ## 0.3.0 is a clean break
 
 TurnKit 0.3.0 intentionally removes the short-lived legacy names from the 0.2