riffer 0.46.1 → 0.47.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 569f1c5929847037c97cc5552eb38a8609db6a1e67ce99b8135961d50aa7a0b5
-  data.tar.gz: c8e03583eb04ff2c8922b8d554c10f5a47fb4421b15600905ba21b0d69031aed
+  metadata.gz: 63cc9b38354166dc44a162e7fb8cc168cf14e950d8cbbba582fb9710721c25a5
+  data.tar.gz: 8a386abc38526d731d7253efbab2c8e2ab345da712762d605f469c5708ec5cca
 SHA512:
-  metadata.gz: 6a0380696a959b4c291a5bf52c03eb7a1e06971a60ded5d058311bbdfe9efd99e58bf94506980e2d4b7b0e21257773894d11be4d1a896ea1e7e3ba8e2ce3359e
-  data.tar.gz: 965f4d1fe6b97f4bd887a1fddd310c7a4cb81b5a0789cf8b91c58c49796abe79f396ec410b6008895275bd16281593a29f53d870e591124ade85b38c62a8fb22
+  metadata.gz: 9a36b54ef99b21ea74a8e215c3ba5abb70c2bf949e40bf15056209f9cc4875c62505406568fc5a258508d17735a5d92c360510a5092a4f158733feebd6f64fb4
+  data.tar.gz: 69209c501af21331079d0e5374b4b41119619ddd81afa6debeb8e93da3955bb144370cb3c1b506411043de9544f070855c66ef62819d5affadc3d671410b906f

data/.release-please-manifest.json

@@ -1,3 +1,3 @@
 {
-  ".": "0.46.1"
+  ".": "0.47.0"
 }

data/CHANGELOG.md

@@ -5,6 +5,13 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.47.0](https://github.com/janeapp/riffer/compare/riffer/v0.46.1...riffer/v0.47.0) (2026-09-11)
+
+
+### Features
+
+* **bedrock:** moving cachePoint so the conversation tail is cached ([#431](https://github.com/janeapp/riffer/issues/431)) ([4fa44b8](https://github.com/janeapp/riffer/commit/4fa44b817c00642303ac30002c33dd77c6d502ab))
+
 ## [0.46.1](https://github.com/janeapp/riffer/compare/riffer/v0.46.0...riffer/v0.46.1) (2026-09-11)
 
 

data/docs/providers/AMAZON_BEDROCK.md

@@ -94,7 +94,10 @@ model_options additional_model_request_fields: {
 
 ### cache_control
 
-Enable prompt caching for models that support it (Claude, Nova). Riffer appends a single Converse `cachePoint` to the stable prefix — after the system array, or after the tools when there is no system prompt — so system instructions and tool definitions are reused across the calls in an agent loop and across conversation turns. The volatile message tail is never cached.
+Enable prompt caching for models that support it (Claude, Nova). Riffer mirrors Anthropic's automatic caching with two Converse `cachePoint` blocks:
+
+- A **static** checkpoint at the end of the stable prefix — after the system array, or after the tools when there is no system prompt — so system instructions and tool definitions are reused across every call.
+- A **moving** checkpoint after the last content block of the final message (a user turn or a batch of tool results). Bedrock looks back roughly 20 content blocks from a checkpoint for the longest cached prefix, so the second and later calls in a tool loop, and later turns in a conversation, read the accumulated conversation from cache instead of re-billing it at the full input rate.
 
 ```ruby
 # 5-minute TTL (default)
@@ -104,7 +107,7 @@ model_options cache_control: {type: "ephemeral"}
 model_options cache_control: {type: "ephemeral", ttl: "1h"}
 ```
 
-Caching is opt-in: omit `cache_control` and no cachePoint is sent. The breakpoint is only honored once the prefix clears the model's minimum token count; on models that don't support `cachePoint`, the Converse request errors. Verify hits via `response.token_usage.cache_read_tokens`.
+Both checkpoints share the same `ttl`. Caching is opt-in: omit `cache_control` and no cachePoint is sent. A checkpoint is only honored once the content before it clears the model's minimum token count (see the per-model limits in the [AWS prompt caching guide](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html)) and is silently ignored below it, so short conversations may see no cache reads at first. On models that don't support `cachePoint`, the Converse request errors. Verify hits via `response.token_usage.cache_read_tokens`, which should grow with each step of an agent loop as the conversation accumulates.
 
 ## Example
 

data/lib/riffer/providers/amazon_bedrock.rb

@@ -129,21 +129,25 @@ class Riffer::Providers::AmazonBedrock < Riffer::Providers::Base
     params
   end
 
-  # Converse chains +tools -> system -> messages+, so a single +cachePoint+ at
-  # the end of the system array (or the tools array, when there is no system
-  # prompt) also caches the preceding sections.
+  # Converse treats +tools -> system -> messages+ as one prefix and looks back
+  # from a +cachePoint+ for the longest cached run, so the point on the final
+  # message reuses the previous step's cache wherever that point sat. Mixed
+  # ttls must be ordered 1h before 5m, so both points share one.
   #--
   #: (Hash[Symbol, untyped], untyped) -> void
   def apply_cache_point(params, cache_control)
     cache_point = { cache_point: build_cache_point(cache_control) }
     system = params[:system]
     tools = params.dig(:tool_config, :tools)
+    last_message = params[:messages].last
 
     if system && !system.empty?
       system << cache_point
     elsif tools && !tools.empty?
       tools << cache_point
     end
+
+    last_message[:content] << cache_point if last_message
   end
 
   #--

data/lib/riffer/version.rb

@@ -2,5 +2,5 @@
 # rbs_inline: enabled
 
 module Riffer
-  VERSION = "0.46.1" #: String
+  VERSION = "0.47.0" #: String
 end

data/sig/generated/riffer/providers/amazon_bedrock.rbs

@@ -45,9 +45,10 @@ class Riffer::Providers::AmazonBedrock < Riffer::Providers::Base
   # : (Array[Riffer::Messages::Base], String?, Hash[Symbol, untyped]) -> Hash[Symbol, untyped]
   def build_request_params: (Array[Riffer::Messages::Base], String?, Hash[Symbol, untyped]) -> Hash[Symbol, untyped]
 
-  # Converse chains +tools -> system -> messages+, so a single +cachePoint+ at
-  # the end of the system array (or the tools array, when there is no system
-  # prompt) also caches the preceding sections.
+  # Converse treats +tools -> system -> messages+ as one prefix and looks back
+  # from a +cachePoint+ for the longest cached run, so the point on the final
+  # message reuses the previous step's cache wherever that point sat. Mixed
+  # ttls must be ordered 1h before 5m, so both points share one.
   # --
   # : (Hash[Symbol, untyped], untyped) -> void
   def apply_cache_point: (Hash[Symbol, untyped], untyped) -> void

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: riffer
 version: !ruby/object:Gem::Version
-  version: 0.46.1
+  version: 0.47.0
 platform: ruby
 authors:
 - Jake Bottrall