riffer 0.16.1 → 0.17.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: dd7f9f22c84f8d3680662d9cac4b242f0425395f510b36654c910b91448dc34f
-  data.tar.gz: 1718dd3b5ca59e62f3287e7ecdfdff201de124bcd2230687a7e3d5ed554fdc95
+  metadata.gz: 849df927faaf4614ad3a4833aafa5323731d3c1f4a7a8687db35ee5d341eb334
+  data.tar.gz: 1d992d83487a673fef8714d6f72811eb94fdb5e646e33612eb18ac915257b832
 SHA512:
-  metadata.gz: bda15f6aef23966f078cbae8be85dce02f177d9d51cc8fb9c08368dc113dbac8efe6ecbe356ae22a5eba043a98278622f1ddc10126eaada061b98eec849ea75c
-  data.tar.gz: 02f951011443b6ce726292be3f19ceb870b1177c03a3eb78b876170aef4d667b583def11e0242acee130c6e87654c262a0f91df366daf537060be2f88aa33283
+  metadata.gz: f1967690dfbe21181d4670117de7563936c05dd7d7303597378584acba35a0543de292fb9e4f72d9ed71789d4b73dc17023ce1d1c0bba27434dea93646dccc4d
+  data.tar.gz: 71b4c1568a2b4f5f420f3546283f5c56bc979b8234734049a8df714363faf132f36bc892dabc3e62702c59c43e74ad7e697fb702c772f2bfdec6f56950c0a1fb

data/.agents/architecture.md

@@ -16,6 +16,17 @@ agent = EchoAgent.new
 puts agent.generate('Hello world')
 ```
 
+`instructions` also accepts a Proc for dynamic instructions resolved at generate time. The Proc receives the `context` hash:
+
+```ruby
+class PersonalAgent < Riffer::Agent
+  model 'openai/gpt-5-mini'
+  instructions ->(context) { "You are assisting #{context[:name]}" }
+end
+
+PersonalAgent.generate('Hello!', context: { name: 'Jane' })
+```
+
 ### Providers (`lib/riffer/providers/`)
 
 Adapters for LLM APIs. The base class uses a template-method pattern — `generate_text` and `stream_text` orchestrate the flow, delegating to five hook methods each provider implements:

data/.agents/code-style.md

@@ -30,6 +30,13 @@ end
 - Explain **why** something is done, not **what** is being done
 - Comments should add value beyond what the code already expresses
 
+## Hash Key Convention
+
+- Use **symbol keys** for all internal hashes
+- Use `JSON.parse(str, symbolize_names: true)` at parse boundaries — never `JSON.parse` followed by `transform_keys(&:to_sym)`
+- String keys are only used at serialization boundaries (JSON Schema output, external API payloads)
+- Do not write dual-access patterns like `hash[:key] || hash["key"]` — normalize to symbol keys at the boundary instead
+
 ## Module Structure
 
 ```ruby

data/.agents/rbs-inline.md

@@ -64,8 +64,8 @@ def evaluate(input:, output:)
 def evaluate(input:, output:, context: nil)
 
 # Positional + keyword parameters
-#: (String, ?tool_context: Hash[Symbol, untyped]?) -> String
-def generate(prompt, tool_context: nil)
+#: (String, ?context: Hash[Symbol, untyped]?) -> String
+def generate(prompt, context: nil)
 
 # Splat/double-splat
 #: (**untyped) -> void

data/.release-please-manifest.json

@@ -1,3 +1,3 @@
 {
-  ".": "0.16.1"
+  ".": "0.17.0"
 }

data/CHANGELOG.md

@@ -5,6 +5,30 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.17.0](https://github.com/janeapp/riffer/compare/riffer/v0.16.1...riffer/v0.17.0) (2026-03-06)
+
+
+### ⚠ BREAKING CHANGES
+
+* rename `tool_context` to `context` ([#159](https://github.com/janeapp/riffer/issues/159))
+
+### Features
+
+* add experimental ToolRuntime abstraction for tool execution ([#156](https://github.com/janeapp/riffer/issues/156)) ([0ca7563](https://github.com/janeapp/riffer/commit/0ca7563df9f0a555e5fa6a1f3065d5f072abbf7e))
+* add interrupt! public method for clean loop interrupts ([#155](https://github.com/janeapp/riffer/issues/155)) ([a4cc877](https://github.com/janeapp/riffer/commit/a4cc8778b754e3932748454446eebd71795ad5e1))
+* add support for dynamic instructions ([#158](https://github.com/janeapp/riffer/issues/158)) ([408e09c](https://github.com/janeapp/riffer/commit/408e09c585142caca173a232ec20dde012553dc0))
+* auto-derive step offset on resume for max_steps enforcement ([#154](https://github.com/janeapp/riffer/issues/154)) ([fb97dbe](https://github.com/janeapp/riffer/commit/fb97dbec4a0edf5ea1e46bf44b93170663db04ef))
+
+
+### Bug Fixes
+
+* resolve edge cases in generate/resume and streaming methods ([#162](https://github.com/janeapp/riffer/issues/162)) ([f74d373](https://github.com/janeapp/riffer/commit/f74d373fb3cb8bb2d6c4617dd29ba3b30a3a8177))
+
+
+### Code Refactoring
+
+* rename `tool_context` to `context` ([#159](https://github.com/janeapp/riffer/issues/159)) ([5be7214](https://github.com/janeapp/riffer/commit/5be7214934866dfa24062b248c59324178f9956a))
+
 ## [0.16.1](https://github.com/janeapp/riffer/compare/riffer/v0.16.0...riffer/v0.16.1) (2026-03-03)
 
 

data/docs/03_AGENTS.md

@@ -37,12 +37,12 @@ class MyAgent < Riffer::Agent
 end
 ```
 
-When the lambda accepts a parameter, it receives the `tool_context`:
+When the lambda accepts a parameter, it receives the `context`:
 
 ```ruby
 class MyAgent < Riffer::Agent
-  model ->(ctx) {
-    ctx&.dig(:premium) ? "anthropic/claude-sonnet-4-20250514" : "anthropic/claude-haiku-4-5-20251001"
+  model ->(context) {
+    context&.dig(:premium) ? "anthropic/claude-sonnet-4-20250514" : "anthropic/claude-haiku-4-5-20251001"
   }
 end
 ```
@@ -60,6 +60,28 @@ class MyAgent < Riffer::Agent
 end
 ```
 
+Instructions can also be resolved dynamically with a lambda:
+
+```ruby
+class MyAgent < Riffer::Agent
+  model 'openai/gpt-4o'
+  instructions -> { "Today is #{Date.today}. You are a helpful assistant." }
+end
+```
+
+When the lambda accepts a parameter, it receives the `context`:
+
+```ruby
+class MyAgent < Riffer::Agent
+  model 'openai/gpt-4o'
+  instructions ->(ctx) { "You are assisting #{ctx[:name]}" }
+end
+
+MyAgent.generate('Hello!', context: { name: 'Jane' })
+```
+
+The lambda is re-evaluated on each `generate` or `stream` call, so instructions can change between calls based on runtime context.
+
 ### identifier
 
 Sets a custom identifier (defaults to snake_case class name):
@@ -219,6 +241,22 @@ Using both `of:` and a block raises `Riffer::ArgumentError`. Using `of:` with a
 
 Structured output is not compatible with streaming — calling `stream` on an agent with structured output configured raises `Riffer::ArgumentError`.
 
+### tool_runtime (Experimental)
+
+> **Warning:** This feature is experimental and may be removed or changed without warning in a future release.
+
+Configures how tool calls are executed. Defaults to sequential (inline) execution:
+
+```ruby
+class MyAgent < Riffer::Agent
+  model 'openai/gpt-4o'
+  uses_tools [WeatherTool, SearchTool]
+  tool_runtime Riffer::ToolRuntime::Threaded
+end
+```
+
+Accepts a `Riffer::ToolRuntime` subclass, a `Riffer::ToolRuntime` instance, or a `Proc`. Inherited by subclasses. When unset, falls back to `Riffer.config.tool_runtime`. See [Tools — Tool Runtime](04_TOOLS.md#tool-runtime-experimental) for details.
+
 ### guardrail
 
 Registers guardrails for pre/post processing of messages. Pass the guardrail class and any options:
@@ -266,8 +304,8 @@ response = MyAgent.generate([
   {role: 'user', content: 'How are you?'}
 ])
 
-# With tool context
-response = MyAgent.generate('Look up my orders', tool_context: {user_id: 123})
+# With context
+response = MyAgent.generate('Look up my orders', context: {user_id: 123})
 
 # With files (string prompt + files shorthand)
 response = MyAgent.generate('What is in this image?', files: [
@@ -352,17 +390,17 @@ Works with both `generate` and `stream`. Only emits agent-generated messages (As
 
 #### Interrupting the Agent Loop
 
-Callbacks can interrupt the agent loop using Ruby's `throw`/`catch` pattern. This is useful for human-in-the-loop approval, cost limits, or content filtering.
+Callbacks can interrupt the agent loop. This is useful for human-in-the-loop approval, cost limits, or content filtering.
 
-Use `throw :riffer_interrupt` to stop the loop. The response will have `interrupted?` set to `true` and contain the accumulated content up to the point of interruption.
+Use `agent.interrupt!` (or the lower-level `throw :riffer_interrupt`) to stop the loop. The response will have `interrupted?` set to `true` and contain the accumulated content up to the point of interruption.
 
-An optional reason can be passed as the second argument to `throw`. It is available via `interrupt_reason` on the response (generate) or `reason` on the `Interrupt` event (stream):
+An optional reason can be passed to `interrupt!`. It is available via `interrupt_reason` on the response (generate) or `reason` on the `Interrupt` event (stream):
 
 ```ruby
 agent = MyAgent.new
 agent.on_message do |msg|
   if msg.is_a?(Riffer::Messages::Tool)
-    throw :riffer_interrupt, "needs human approval"
+    agent.interrupt!("needs human approval")
   end
 end
 
@@ -410,7 +448,7 @@ For cross-process resume (e.g., after a process restart or async approval), pass
 # Persist messages during generation (e.g., via on_message callback)
 # Later, in a new process:
 agent = MyAgent.new
-response = agent.resume(messages: persisted_messages, tool_context: {user_id: 123})
+response = agent.resume(messages: persisted_messages, context: {user_id: 123})
 
 # Or resume in streaming mode:
 agent.resume_stream(messages: persisted_messages).each do |event|
@@ -429,7 +467,7 @@ Continues an agent loop synchronously. Returns a `Riffer::Agent::Response` objec
 response = agent.resume
 
 # Cross-process resume from persisted messages
-response = agent.resume(messages: persisted_messages, tool_context: {user_id: 123})
+response = agent.resume(messages: persisted_messages, context: {user_id: 123})
 ```
 
 ### resume_stream
@@ -449,6 +487,16 @@ agent.resume_stream(messages: persisted_messages).each do |event|
 end
 ```
 
+### interrupt!
+
+Interrupts the agent loop from an `on_message` callback. Equivalent to `throw :riffer_interrupt, reason`:
+
+```ruby
+agent.on_message do |msg|
+  agent.interrupt!(:needs_approval) if requires_approval?(msg)
+end
+```
+
 ### token_usage
 
 Access cumulative token usage across all LLM calls:
@@ -532,7 +580,7 @@ end
 When an agent receives a response with tool calls:
 
 1. Agent detects `tool_calls` in the assistant message
-2. For each tool call:
+2. The configured tool runtime executes the tool calls (sequentially by default, or concurrently with `Riffer::ToolRuntime::Threaded`):
    - Finds the matching tool class
    - Validates arguments against the tool's parameter schema
    - Calls the tool's `call` method with `context` and arguments
@@ -576,7 +624,7 @@ response.tripwire.reason   # => "Content policy violation"
 
 ### Callback Interrupt (imperative, external)
 
-Callbacks registered with `on_message` can call `throw :riffer_interrupt` to pause the loop at any point — after receiving an assistant message, after a tool result, etc. The caller controls exactly when and why to interrupt.
+Callbacks registered with `on_message` can call `agent.interrupt!` (or `throw :riffer_interrupt`) to pause the loop at any point — after receiving an assistant message, after a tool result, etc. The caller controls exactly when and why to interrupt.
 
 - **When to use:** Flow control that depends on runtime decisions — human-in-the-loop approval, budget tracking, conditional pausing.
 - **Response:** `response.interrupted?` returns `true`, `response.interrupt_reason` contains the optional reason.
@@ -586,7 +634,7 @@ Callbacks registered with `on_message` can call `throw :riffer_interrupt` to pau
 ```ruby
 agent = MyAgent.new
 agent.on_message do |msg|
-  throw :riffer_interrupt, "approval needed" if requires_approval?(msg)
+  agent.interrupt!("approval needed") if requires_approval?(msg)
 end
 
 response = agent.generate('Do something risky')

data/docs/04_TOOLS.md

@@ -145,7 +145,7 @@ Every tool must implement the `call` method and return a `Riffer::Tools::Respons
 
 ```ruby
 def call(context:, **kwargs)
-  # context - The tool_context passed to agent.generate()
+  # context - The context passed to agent.generate()
   # kwargs  - Validated parameters
   #
   # Must return a Riffer::Tools::Response
@@ -154,7 +154,7 @@ end
 
 ### Accessing Context
 
-The `context` argument receives whatever was passed to `tool_context`:
+The `context` argument receives whatever was passed as `context:` to `generate`:
 
 ```ruby
 class UserOrdersTool < Riffer::Tool
@@ -172,7 +172,7 @@ class UserOrdersTool < Riffer::Tool
 end
 
 # Usage
-agent.generate("Show my orders", tool_context: {user_id: 123})
+agent.generate("Show my orders", context: {user_id: 123})
 ```
 
 ## Response Objects
@@ -368,6 +368,127 @@ rescue => e
 end
 ```
 
-Unhandled exceptions are caught by Riffer and converted to error responses with type `:execution_error`. However, it's recommended to handle expected errors explicitly for better error messages.
+Unhandled `RuntimeError` exceptions are caught by Riffer and converted to error responses with type `:execution_error`. For expected execution errors, raise `Riffer::ToolExecutionError` — these are also caught and returned to the LLM. Programming bugs (`NoMethodError`, `NameError`, `TypeError`, etc.) propagate to the caller. It's recommended to handle expected errors explicitly for better error messages.
 
 The LLM receives the error message and can decide how to respond (retry, apologize, ask for different input, etc.).
+
+## Tool Runtime (Experimental)
+
+> **Warning:** This feature is experimental and may be removed or changed without warning in a future release.
+
+By default, tool calls are executed sequentially in the current thread using `Riffer::ToolRuntime::Inline`. You can change how tool calls are executed by configuring a different tool runtime.
+
+### Built-in Runtimes
+
+| Runtime | Description |
+|---------|-------------|
+| `Riffer::ToolRuntime::Inline` | Executes tool calls sequentially (default) |
+| `Riffer::ToolRuntime::Threaded` | Executes tool calls concurrently using threads |
+
+### Per-Agent Configuration
+
+Use the `tool_runtime` class method on your agent:
+
+```ruby
+class MyAgent < Riffer::Agent
+  model 'openai/gpt-4o'
+  uses_tools [WeatherTool, SearchTool]
+  tool_runtime Riffer::ToolRuntime::Threaded
+end
+```
+
+Accepted values:
+
+- A `Riffer::ToolRuntime` subclass — instantiated automatically (e.g., `Riffer::ToolRuntime::Inline`, `Riffer::ToolRuntime::Threaded`)
+- A `Riffer::ToolRuntime` instance — for custom runtimes with specific options
+- A `Proc` — evaluated at runtime (see below)
+
+### Dynamic Resolution
+
+Use a lambda for context-aware runtime selection:
+
+```ruby
+class MyAgent < Riffer::Agent
+  model 'openai/gpt-4o'
+  uses_tools [WeatherTool, SearchTool]
+
+  tool_runtime ->(context) {
+    context&.dig(:parallel) ? Riffer::ToolRuntime::Threaded.new : Riffer::ToolRuntime::Inline.new
+  }
+end
+
+agent.generate("Do work", context: {parallel: true})
+```
+
+When the lambda accepts a parameter, it receives the `context`. Zero-arity lambdas are also supported.
+
+### Global Configuration
+
+Set a default tool runtime for all agents:
+
+```ruby
+Riffer.configure do |config|
+  config.tool_runtime = Riffer::ToolRuntime::Threaded
+end
+```
+
+Per-agent configuration overrides the global default.
+
+### Threaded Runtime Considerations
+
+When using `Riffer::ToolRuntime::Threaded`, each tool call runs in its own thread. The `around_tool_call` hook also runs inside that thread. Be mindful of thread-local state — for example, `ActiveRecord::Base.connection`, `RequestStore`, or any `Thread.current[]` values may not be available or may behave differently across threads. Ensure your tools and hooks are thread-safe.
+
+### Threaded Runtime Options
+
+The threaded runtime accepts a `max_concurrency` option (default: 5):
+
+```ruby
+class MyAgent < Riffer::Agent
+  model 'openai/gpt-4o'
+  uses_tools [WeatherTool, SearchTool]
+  tool_runtime Riffer::ToolRuntime::Threaded.new(max_concurrency: 3)
+end
+```
+
+### Custom Runtimes
+
+Create a custom runtime by subclassing `Riffer::ToolRuntime` and overriding the private `dispatch_tool_call` method:
+
+```ruby
+class HttpToolRuntime < Riffer::ToolRuntime
+  private
+
+  def dispatch_tool_call(tool_call, tools:, context:)
+    # Dispatch tool execution to an external service
+    response = HttpClient.post("/tools/execute", {
+      name: tool_call.name,
+      arguments: tool_call.arguments
+    })
+    Riffer::Tools::Response.text(response.body)
+  rescue Riffer::ToolExecutionError => e
+    Riffer::Tools::Response.error(e.message, type: :execution_error)
+  rescue RuntimeError => e
+    Riffer::Tools::Response.error("Error executing tool: #{e.message}", type: :execution_error)
+  end
+end
+```
+
+### Around-Call Hook
+
+Each tool call is wrapped by the `around_tool_call` method, which yields by default. Override it in a subclass to add instrumentation, logging, or other cross-cutting concerns:
+
+```ruby
+class InstrumentedRuntime < Riffer::ToolRuntime::Inline
+  private
+
+  def around_tool_call(tool_call, context:)
+    start = Time.now
+    result = yield
+    duration = Time.now - start
+    Rails.logger.info("Tool #{tool_call.name} took #{duration}s")
+    result
+  end
+end
+```
+
+Subclasses inherit the hook and can override it further.

data/docs/06_STREAM_EVENTS.md

@@ -190,7 +190,7 @@ See [Guardrails](09_GUARDRAILS.md) for more information.
 
 Emitted when the agent loop is interrupted. This can happen in two ways:
 
-- An `on_message` callback calls `throw :riffer_interrupt` (reason is a String or `nil`).
+- An `on_message` callback calls `agent.interrupt!` or `throw :riffer_interrupt` (reason is a String or `nil`).
 - The `max_steps` limit is reached (reason is the Symbol `:max_steps`).
 
 This is the streaming equivalent of `Response#interrupted?` in generate mode.

data/docs/07_CONFIGURATION.md

@@ -72,6 +72,26 @@ Riffer.configure do |config|
 end
 ```
 
+### Tool Runtime (Experimental)
+
+> **Warning:** This feature is experimental and may be removed or changed without warning in a future release.
+
+Configure the default tool runtime for all agents:
+
+```ruby
+Riffer.configure do |config|
+  config.tool_runtime = Riffer::ToolRuntime::Threaded
+end
+```
+
+| Value | Description |
+|-------|-------------|
+| `Riffer::ToolRuntime` subclass | Instantiated automatically (e.g., `Riffer::ToolRuntime::Inline`, `Riffer::ToolRuntime::Threaded`) |
+| `Riffer::ToolRuntime` instance | Custom runtime with specific options |
+| `Proc` | Dynamic resolution |
+
+Per-agent configuration overrides this global default. See [Tools — Tool Runtime](04_TOOLS.md#tool-runtime-experimental) for details.
+
 ## Agent-Level Configuration
 
 Override global configuration at the agent level:

data/docs/08_EVALS.md

@@ -81,23 +81,23 @@ result = Riffer::Evals::EvaluatorRunner.run(
 )
 ```
 
-### Tool Context
+### Context
 
-Pass `tool_context:` to provide context that agents use for dynamic model selection, tool resolution, or tool execution:
+Pass `context:` to provide context that agents use for dynamic model selection, tool resolution, or tool execution:
 
 ```ruby
 result = Riffer::Evals::EvaluatorRunner.run(
   agent: MyAgent,
   scenarios: [
     { input: "What is Ruby?" },
-    { input: "Premium question", tool_context: { premium: true } }
+    { input: "Premium question", context: { premium: true } }
   ],
   evaluators: [AnswerRelevancyEvaluator],
-  tool_context: { premium: false }
+  context: { premium: false }
 )
 ```
 
-Per-scenario `tool_context` overrides the top-level value. Scenarios without their own `tool_context` inherit the top-level value.
+Per-scenario `context` overrides the top-level value. Scenarios without their own `context` inherit the top-level value.
 
 ### RunResult
 

data/docs_providers/05_MOCK_PROVIDER.md

@@ -121,7 +121,7 @@ class MyAgentTest < Minitest::Test
     ])
     @provider.stub_response("Done.")
 
-    @agent.generate("Do something", tool_context: {user_id: 123})
+    @agent.generate("Do something", context: {user_id: 123})
 
     # Tool receives the context
   end