raif 1.1.0 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1af0a7a003990a40716f3d07ec87838e5fb485147bb87a9b35ae1fe23f642501
-  data.tar.gz: b82bba67ed23a6e47e1bd02f5c64f9448c7374017c097a012ccdae2e3795be34
+  metadata.gz: 93d75d28d64da2a5559de740890291cb3a125d981bb58aa56ff4c45be0169954
+  data.tar.gz: 9f3ca2e5142f43be17c9fe27e3572cd08a4241e72d6e43893b74e6ba954169ae
 SHA512:
-  metadata.gz: d9dd7273eeccb284d7ee720fe586dc1e86b6e3355da378a719310cbe3209c508a57a3ee1cfcaa60762cd2860f76960b4f89aa39d7f5eafa90eb97b78adab6123
-  data.tar.gz: 26013bb1beb60367d878b451163594c50e36ef046b8ae0864a4728c4d0ae862e79ec22ab5ccb6c0acd88991b9f227b3af4b31daedc76e90755e9d87bce45f25c
+  metadata.gz: 90726f18f49a312f0e8d2ceb3913c1bb45c50ad237f0fceeefa825e28374214a8c61ae84c4a83e6d3188a27adea3dc0162001792d867babadca5e30f5f736398
+  data.tar.gz: e053003fd9c510509cbc749d5f20d9d2596396224499fe7cc40467920a216f4da6efb3c281ed451ff7e29bced98ff17476c4779b7760a3e361c4a09285667deb

data/README.md

@@ -13,16 +13,21 @@ Raif is built by [Cultivate Labs](https://www.cultivatelabs.com) and is used to
 ## Table of Contents
 - [Setup](#setup)
   - [OpenAI](#openai)
+    - [OpenAI Completions API](#openai-completions-api)
+    - [OpenAI Responses API](#openai-responses-api)
   - [Anthropic Claude](#anthropic-claude)
   - [AWS Bedrock (Claude)](#aws-bedrock-claude)
   - [OpenRouter](#openrouter)
 - [Chatting with the LLM](#chatting-with-the-llm)
+  - [Streaming Responses](#streaming-responses)
 - [Key Raif Concepts](#key-raif-concepts)
   - [Tasks](#tasks)
   - [Conversations](#conversations)
+    - [Real-time Streaming Responses](#real-time-streaming-responses)
     - [Conversation Types](#conversation-types)
   - [Agents](#agents)
   - [Model Tools](#model-tools)
+    - [Provider-Managed Tools](#provider-managed-tools)
 - [Images/Files/PDF's](#imagesfilespdfs)
   - [Images/Files/PDF's in Tasks](#imagesfilespdfs-in-tasks)
 - [Embedding Models](#embedding-models)
@@ -35,6 +40,7 @@ Raif is built by [Cultivate Labs](https://www.cultivatelabs.com) and is used to
   - [Adding LLM Models](#adding-llm-models)
 - [Testing](#testing)
 - [Demo App](#demo-app)
+- [Contributing](#contributing)
 - [License](#license)
 
 # Setup
@@ -60,6 +66,8 @@ This will:
 - Copy Raif's database migrations to your application
 - Mount Raif's engine at `/raif` in your application's `config/routes.rb` file
 
+You must configure at least one API key for your LLM provider ([OpenAI](#openai), [Anthropic Claude](#anthropic-claude), [AWS Bedrock](#aws-bedrock-claude), [OpenRouter](#openrouter)). By default, the initializer will load them from environment variables (e.g. `ENV["OPENAI_API_KEY"]`, `ENV["ANTHROPIC_API_KEY"]`, `ENV["OPENROUTER_API_KEY"]`). Alternatively, you can set them directly in `config/initializers/raif.rb`.
+
 Run the migrations. Raif is compatible with both PostgreSQL and MySQL databases.
 ```bash
 rails db:migrate
@@ -82,6 +90,10 @@ end
 Configure your LLM providers. You'll need at least one of:
 
 ## OpenAI
+
+Raif supports both OpenAI's [Completions API](https://platform.openai.com/docs/api-reference/chat) and the newer [Responses API](https://platform.openai.com/docs/api-reference/responses), which provides access to provider-managed tools like web search, code execution, and image generation.
+
+### OpenAI Completions API
 ```ruby
 Raif.configure do |config|
   config.open_ai_models_enabled = true
@@ -90,10 +102,44 @@ Raif.configure do |config|
 end
 ```
 
-Currently supported OpenAI models:
+Currently supported OpenAI Completions API models:
 - `open_ai_gpt_4o_mini`
 - `open_ai_gpt_4o`
 - `open_ai_gpt_3_5_turbo`
+- `open_ai_gpt_4_1`
+- `open_ai_gpt_4_1_mini`
+- `open_ai_gpt_4_1_nano`
+- `open_ai_o1`
+- `open_ai_o1_mini`
+- `open_ai_o3`
+- `open_ai_o3_mini`
+- `open_ai_o4_mini`
+
+### OpenAI Responses API
+```ruby
+Raif.configure do |config|
+  config.open_ai_models_enabled = true
+  config.open_ai_api_key = ENV["OPENAI_API_KEY"]
+  config.default_llm_model_key = "open_ai_responses_gpt_4o"
+end
+```
+
+Currently supported OpenAI Responses API models:
+- `open_ai_responses_gpt_4o_mini`
+- `open_ai_responses_gpt_4o`
+- `open_ai_responses_gpt_3_5_turbo`
+- `open_ai_responses_gpt_4_1`
+- `open_ai_responses_gpt_4_1_mini`
+- `open_ai_responses_gpt_4_1_nano`
+- `open_ai_responses_o1`
+- `open_ai_responses_o1_mini`
+- `open_ai_responses_o1_pro`
+- `open_ai_responses_o3`
+- `open_ai_responses_o3_mini`
+- `open_ai_responses_o3_pro`
+- `open_ai_responses_o4_mini`
+
+The Responses API provides access to [provider-managed tools](#provider-managed-tools), including web search, code execution, and image generation.
 
 ## Anthropic Claude
 ```ruby
@@ -110,10 +156,12 @@ Currently supported Anthropic models:
 - `anthropic_claude_3_5_haiku`
 - `anthropic_claude_3_opus`
 
+The Anthropic adapter provides access to [provider-managed tools](#provider-managed-tools) for web search and code execution.
+
 ## AWS Bedrock (Claude)
 ```ruby
 Raif.configure do |config|
-  config.anthropic_bedrock_models_enabled = true
+  config.bedrock_models_enabled = true
   config.aws_bedrock_region = "us-east-1"
   config.default_llm_model_key = "bedrock_claude_3_5_sonnet"
 end
@@ -124,6 +172,9 @@ Currently supported Bedrock models:
 - `bedrock_claude_3_7_sonnet`
 - `bedrock_claude_3_5_haiku`
 - `bedrock_claude_3_opus`
+- `bedrock_amazon_nova_micro`
+- `bedrock_amazon_nova_lite`
+- `bedrock_amazon_nova_pro`
 
 Note: Raif utilizes the [AWS Bedrock gem](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/BedrockRuntime/Client.html) and AWS credentials should be configured via the AWS SDK (environment variables, IAM role, etc.)
 
@@ -184,6 +235,38 @@ puts model_completion.parsed_response # will strip backticks, parse the JSON, an
 # => {"joke" => "Why don't skeletons fight each other? They don't have the guts."}
 ```
 
+## Streaming Responses
+
+You can enable streaming for any chat call by passing a block to the `chat` method. When streaming is enabled, the block will be called with partial responses as they're received from the LLM:
+
+```ruby
+llm = Raif.llm(:open_ai_gpt_4o)
+model_completion = llm.chat(message: "Tell me a story") do |model_completion, delta, sse_event|
+  # This block is called multiple times as the response streams in.
+  # You could broadcast these updates via Turbo Streams, WebSockets, etc.
+  Turbo::StreamsChannel.broadcast_replace_to(
+    :my_channel,
+    target: "chat-response",
+    partial: "my_partial_displaying_chat_response",
+    locals: { model_completion: model_completion, delta: delta, sse_event: sse_event }
+  )
+end
+
+# The final complete response is available in the model_completion
+puts model_completion.raw_response
+```
+
+You can configure the streaming update frequency by adjusting the chunk size threshold in your Raif configuration:
+
+```ruby
+Raif.configure do |config|
+  # Control how often the model completion is updated & the block is called when streaming.
+  # Lower values = more frequent updates but more database writes.
+  # Higher values = less frequent updates but fewer database writes.
+  config.streaming_update_chunk_size_threshold = 50 # default is 25
+end
+```
+
 # Key Raif Concepts
 
 ## Tasks
@@ -334,6 +417,10 @@ If your app already includes Bootstrap styles, this will render a conversation i
 
 If your app does not include Bootstrap, you can [override the views](#views) to update styles.
 
+### Real-time Streaming Responses
+
+Raif conversations have built-in support for streaming responses, where the LLM's response is displayed progressively as it's being generated. Each time a conversation entry is updated during the streaming response, Raif will call `broadcast_replace_to(conversation)` (where `conversation` is the `Raif::Conversation` associated with the conversation entry). When using the `raif_conversation` view helper, it will automatically set up the subscription for you.
+
 ### Conversation Types
 
 If your application has a specific type of conversation that you use frequently, you can create a custom conversation type by running the generator. For example, say you are implementing a customer support chatbot in your application and want to have a custom conversation type for doing this with the LLM:
@@ -520,7 +607,54 @@ class Raif::ModelTools::GoogleSearch < Raif::ModelTool
 end
 ```
 
-## Images/Files/PDF's
+### Provider-Managed Tools
+
+In addition to the ability to create your own model tools, Raif supports provider-managed tools. These are tools that are built into certain LLM providers and run on the provider's infrastructure:
+
+- **`Raif::ModelTools::ProviderManaged::WebSearch`**: Performs real-time web searches and returns relevant results
+- **`Raif::ModelTools::ProviderManaged::CodeExecution`**: Executes code in a secure sandboxed environment (e.g. Python)
+- **`Raif::ModelTools::ProviderManaged::ImageGeneration`**: Generates images based on text descriptions
+
+Current provider-managed tool support:
+| Provider | WebSearch | CodeExecution | ImageGeneration |
+|----------|-----------|---------------|-----------------|
+| OpenAI Responses API | ✅ | ✅ | ✅ |
+| OpenAI Completions API | ❌ | ❌ | ❌ |
+| Anthropic Claude | ✅ | ✅ | ❌ |
+| AWS Bedrock (Claude) | ❌ | ❌ | ❌ |
+| OpenRouter | ❌ | ❌ | ❌ |
+
+To use provider-managed tools, include them in the `available_model_tools` array:
+
+```ruby
+# In a conversation
+conversation = Raif::Conversation.create!(
+  creator: current_user,
+  available_model_tools: [
+    "Raif::ModelTools::ProviderManaged::WebSearch",
+    "Raif::ModelTools::ProviderManaged::CodeExecution"
+  ]
+)
+
+# In an agent
+agent = Raif::Agents::ReActAgent.new(
+  task: "Search for recent news about AI and create a summary chart",
+  available_model_tools: [
+    "Raif::ModelTools::ProviderManaged::WebSearch",
+    "Raif::ModelTools::ProviderManaged::CodeExecution"
+  ],
+  creator: current_user
+)
+
+# Directly in a chat
+llm = Raif.llm(:open_ai_responses_gpt_4_1)
+model_completion = llm.chat(
+  messages: [{ role: "user", content: "What are the latest developments in Ruby on Rails?" }], 
+  available_model_tools: [Raif::ModelTools::ProviderManaged::WebSearch]
+)
+```
+
+## Sending Images/Files/PDF's to the LLM
 
 Raif supports images, files, and PDF's in the messages sent to the LLM.
 
@@ -596,7 +730,7 @@ Raif supports generation of vector embeddings. You can enable and configure embe
 ```ruby
 Raif.configure do |config|
   config.open_ai_embedding_models_enabled = true
-  config.aws_bedrock_titan_embedding_models_enabled = true
+  config.bedrock_embedding_models_enabled = true
   
   config.default_embedding_model_key = "open_ai_text_embedding_3_small"
 end
@@ -649,6 +783,7 @@ The admin interface contains sections for:
 - Conversations
 - Agents
 - Model Tool Invocations
+- Stats
 
 
 ### Model Completions
@@ -670,6 +805,9 @@ The admin interface contains sections for:
   ![Model Tool Invocations Index](./screenshots/admin-model-tool-invocations-index.png)
   ![Model Tool Invocation Detail](./screenshots/admin-model-tool-invocation-show.png)
 
+### Stats
+  ![Stats](./screenshots/admin-stats.png)
+
 # Customization
 
 ## Controllers
@@ -832,6 +970,12 @@ You can then access the app at [http://localhost:3000](http://localhost:3000).
 
 ![Demo App Screenshot](./screenshots/demo-app.png)
 
+# Contributing
+
+We welcome contributions to Raif! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
+
+**Important**: All PR's should be made against the `dev` branch.
+
 # License
 
 The gem is available as open source under the terms of the MIT License.

data/app/assets/builds/raif.css

@@ -28,6 +28,31 @@
   animation-delay: 0.4s;
 }
 
+.raif-streaming-cursor {
+  display: inline-block;
+  width: 2px;
+  height: 1.1em;
+  margin-bottom: -2px;
+  background-color: currentColor;
+  animation: blink 1s infinite;
+  transform: none;
+  border-radius: 0;
+  position: relative;
+}
+
+.raif-streaming-cursor:before,
+.raif-streaming-cursor:after {
+  display: none;
+}
+
+@keyframes blink {
+  0%, 50% {
+    opacity: 1;
+  }
+  51%, 100% {
+    opacity: 0;
+  }
+}
 @keyframes rotate {
   0% {
     transform: translate(-50%, -50%) rotateZ(0deg);
@@ -71,4 +96,4 @@
   }
 }
 
-/*# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbInJhaWYuY3NzIl0sIm5hbWVzIjpbXSwibWFwcGluZ3MiOiJBQUFBO0VBQ0UseUJBQXlCO0VBQ3pCLG1CQUFtQjtFQUNuQixrQkFBa0I7RUFDbEIsV0FBVztFQUNYLFlBQVk7RUFDWixjQUFjO0VBQ2QscUJBQXFCO0FBQ3ZCOztBQUVBOztFQUVFLFdBQVc7RUFDWCxjQUFjO0VBQ2Qsa0JBQWtCO0VBQ2xCLE1BQU07RUFDTixPQUFPO0VBQ1AsY0FBYztFQUNkLGVBQWU7RUFDZixrQkFBa0I7RUFDbEIseUJBQXlCO0VBQ3pCLGtDQUFrQztBQUNwQzs7QUFFQTtFQUNFLGNBQWM7RUFDZCx5QkFBeUI7RUFDekIscUJBQXFCO0FBQ3ZCOztBQUVBO0VBQ0U7SUFDRSw4Q0FBOEM7RUFDaEQ7RUFDQTtJQUNFLGdEQUFnRDtFQUNsRDtBQUNGO0FBQ0E7RUFDRTtJQUNFLDZDQUE2QztFQUMvQztFQUNBO0lBQ0UsZ0RBQWdEO0VBQ2xEO0FBQ0Y7QUFDQTtFQUNFO0lBQ0Usd0NBQXdDO0VBQzFDO0VBQ0E7SUFDRSx3Q0FBd0M7RUFDMUM7RUFDQTtJQUNFLHNDQUFzQztFQUN4QztFQUNBO0lBQ0UseUNBQXlDO0VBQzNDO0VBQ0E7SUFDRSxxQ0FBcUM7RUFDdkM7RUFDQTtJQUNFLDBDQUEwQztFQUM1QztFQUNBO0lBQ0UsdUNBQXVDO0VBQ3pDO0VBQ0E7SUFDRSx5Q0FBeUM7RUFDM0M7QUFDRiIsImZpbGUiOiJyYWlmLmNzcyIsInNvdXJjZXNDb250ZW50IjpbIi5yYWlmLWxvYWRlciB7XG4gIHRyYW5zZm9ybTogcm90YXRlWig0NWRlZyk7XG4gIHBlcnNwZWN0aXZlOiAxMDAwcHg7XG4gIGJvcmRlci1yYWRpdXM6IDUwJTtcbiAgd2lkdGg6IDI1cHg7XG4gIGhlaWdodDogMjVweDtcbiAgY29sb3I6ICMzODc0ZmY7XG4gIGRpc3BsYXk6IGlubGluZS1ibG9jaztcbn1cblxuLnJhaWYtbG9hZGVyOmJlZm9yZSxcbi5yYWlmLWxvYWRlcjphZnRlciB7XG4gIGNvbnRlbnQ6IFwiXCI7XG4gIGRpc3BsYXk6IGJsb2NrO1xuICBwb3NpdGlvbjogYWJzb2x1dGU7XG4gIHRvcDogMDtcbiAgbGVmdDogMDtcbiAgd2lkdGg6IGluaGVyaXQ7XG4gIGhlaWdodDogaW5oZXJpdDtcbiAgYm9yZGVyLXJhZGl1czogNTAlO1xuICB0cmFuc2Zvcm06IHJvdGF0ZVgoNzBkZWcpO1xuICBhbmltYXRpb246IDFzIHNwaW4gbGluZWFyIGluZmluaXRlO1xufVxuXG4ucmFpZi1sb2FkZXI6YWZ0ZXIge1xuICBjb2xvcjogIzI1YjAwMztcbiAgdHJhbnNmb3JtOiByb3RhdGVZKDcwZGVnKTtcbiAgYW5pbWF0aW9uLWRlbGF5OiAwLjRzO1xufVxuXG5Aa2V5ZnJhbWVzIHJvdGF0ZSB7XG4gIDAlIHtcbiAgICB0cmFuc2Zvcm06IHRyYW5zbGF0ZSgtNTAlLCAtNTAlKSByb3RhdGVaKDBkZWcpO1xuICB9XG4gIDEwMCUge1xuICAgIHRyYW5zZm9ybTogdHJhbnNsYXRlKC01MCUsIC01MCUpIHJvdGF0ZVooMzYwZGVnKTtcbiAgfVxufVxuQGtleWZyYW1lcyByb3RhdGVjY3cge1xuICAwJSB7XG4gICAgdHJhbnNmb3JtOiB0cmFuc2xhdGUoLTUwJSwgLTUwJSkgcm90YXRlKDBkZWcpO1xuICB9XG4gIDEwMCUge1xuICAgIHRyYW5zZm9ybTogdHJhbnNsYXRlKC01MCUsIC01MCUpIHJvdGF0ZSgtMzYwZGVnKTtcbiAgfVxufVxuQGtleWZyYW1lcyBzcGluIHtcbiAgMCUsIDEwMCUge1xuICAgIGJveC1zaGFkb3c6IDAuM2VtIDBweCAwIDBweCBjdXJyZW50Y29sb3I7XG4gIH1cbiAgMTIlIHtcbiAgICBib3gtc2hhZG93OiAwLjNlbSAwLjNlbSAwIDAgY3VycmVudGNvbG9yO1xuICB9XG4gIDI1JSB7XG4gICAgYm94LXNoYWRvdzogMCAwLjNlbSAwIDBweCBjdXJyZW50Y29sb3I7XG4gIH1cbiAgMzclIHtcbiAgICBib3gtc2hhZG93OiAtMC4zZW0gMC4zZW0gMCAwIGN1cnJlbnRjb2xvcjtcbiAgfVxuICA1MCUge1xuICAgIGJveC1zaGFkb3c6IC0wLjNlbSAwIDAgMCBjdXJyZW50Y29sb3I7XG4gIH1cbiAgNjIlIHtcbiAgICBib3gtc2hhZG93OiAtMC4zZW0gLTAuM2VtIDAgMCBjdXJyZW50Y29sb3I7XG4gIH1cbiAgNzUlIHtcbiAgICBib3gtc2hhZG93OiAwcHggLTAuM2VtIDAgMCBjdXJyZW50Y29sb3I7XG4gIH1cbiAgODclIHtcbiAgICBib3gtc2hhZG93OiAwLjNlbSAtMC4zZW0gMCAwIGN1cnJlbnRjb2xvcjtcbiAgfVxufVxuIl19 */
+/*# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbInJhaWYuY3NzIl0sIm5hbWVzIjpbXSwibWFwcGluZ3MiOiJBQUFBO0VBQ0UseUJBQXlCO0VBQ3pCLG1CQUFtQjtFQUNuQixrQkFBa0I7RUFDbEIsV0FBVztFQUNYLFlBQVk7RUFDWixjQUFjO0VBQ2QscUJBQXFCO0FBQ3ZCOztBQUVBOztFQUVFLFdBQVc7RUFDWCxjQUFjO0VBQ2Qsa0JBQWtCO0VBQ2xCLE1BQU07RUFDTixPQUFPO0VBQ1AsY0FBYztFQUNkLGVBQWU7RUFDZixrQkFBa0I7RUFDbEIseUJBQXlCO0VBQ3pCLGtDQUFrQztBQUNwQzs7QUFFQTtFQUNFLGNBQWM7RUFDZCx5QkFBeUI7RUFDekIscUJBQXFCO0FBQ3ZCOztBQUVBO0VBQ0UscUJBQXFCO0VBQ3JCLFVBQVU7RUFDVixhQUFhO0VBQ2IsbUJBQW1CO0VBQ25CLDhCQUE4QjtFQUM5Qiw0QkFBNEI7RUFDNUIsZUFBZTtFQUNmLGdCQUFnQjtFQUNoQixrQkFBa0I7QUFDcEI7O0FBRUE7O0VBRUUsYUFBYTtBQUNmOztBQUVBO0VBQ0U7SUFDRSxVQUFVO0VBQ1o7RUFDQTtJQUNFLFVBQVU7RUFDWjtBQUNGO0FBQ0E7RUFDRTtJQUNFLDhDQUE4QztFQUNoRDtFQUNBO0lBQ0UsZ0RBQWdEO0VBQ2xEO0FBQ0Y7QUFDQTtFQUNFO0lBQ0UsNkNBQTZDO0VBQy9DO0VBQ0E7SUFDRSxnREFBZ0Q7RUFDbEQ7QUFDRjtBQUNBO0VBQ0U7SUFDRSx3Q0FBd0M7RUFDMUM7RUFDQTtJQUNFLHdDQUF3QztFQUMxQztFQUNBO0lBQ0Usc0NBQXNDO0VBQ3hDO0VBQ0E7SUFDRSx5Q0FBeUM7RUFDM0M7RUFDQTtJQUNFLHFDQUFxQztFQUN2QztFQUNBO0lBQ0UsMENBQTBDO0VBQzVDO0VBQ0E7SUFDRSx1Q0FBdUM7RUFDekM7RUFDQTtJQUNFLHlDQUF5QztFQUMzQztBQUNGIiwiZmlsZSI6InJhaWYuY3NzIiwic291cmNlc0NvbnRlbnQiOlsiLnJhaWYtbG9hZGVyIHtcbiAgdHJhbnNmb3JtOiByb3RhdGVaKDQ1ZGVnKTtcbiAgcGVyc3BlY3RpdmU6IDEwMDBweDtcbiAgYm9yZGVyLXJhZGl1czogNTAlO1xuICB3aWR0aDogMjVweDtcbiAgaGVpZ2h0OiAyNXB4O1xuICBjb2xvcjogIzM4NzRmZjtcbiAgZGlzcGxheTogaW5saW5lLWJsb2NrO1xufVxuXG4ucmFpZi1sb2FkZXI6YmVmb3JlLFxuLnJhaWYtbG9hZGVyOmFmdGVyIHtcbiAgY29udGVudDogXCJcIjtcbiAgZGlzcGxheTogYmxvY2s7XG4gIHBvc2l0aW9uOiBhYnNvbHV0ZTtcbiAgdG9wOiAwO1xuICBsZWZ0OiAwO1xuICB3aWR0aDogaW5oZXJpdDtcbiAgaGVpZ2h0OiBpbmhlcml0O1xuICBib3JkZXItcmFkaXVzOiA1MCU7XG4gIHRyYW5zZm9ybTogcm90YXRlWCg3MGRlZyk7XG4gIGFuaW1hdGlvbjogMXMgc3BpbiBsaW5lYXIgaW5maW5pdGU7XG59XG5cbi5yYWlmLWxvYWRlcjphZnRlciB7XG4gIGNvbG9yOiAjMjViMDAzO1xuICB0cmFuc2Zvcm06IHJvdGF0ZVkoNzBkZWcpO1xuICBhbmltYXRpb24tZGVsYXk6IDAuNHM7XG59XG5cbi5yYWlmLXN0cmVhbWluZy1jdXJzb3Ige1xuICBkaXNwbGF5OiBpbmxpbmUtYmxvY2s7XG4gIHdpZHRoOiAycHg7XG4gIGhlaWdodDogMS4xZW07XG4gIG1hcmdpbi1ib3R0b206IC0ycHg7XG4gIGJhY2tncm91bmQtY29sb3I6IGN1cnJlbnRDb2xvcjtcbiAgYW5pbWF0aW9uOiBibGluayAxcyBpbmZpbml0ZTtcbiAgdHJhbnNmb3JtOiBub25lO1xuICBib3JkZXItcmFkaXVzOiAwO1xuICBwb3NpdGlvbjogcmVsYXRpdmU7XG59XG5cbi5yYWlmLXN0cmVhbWluZy1jdXJzb3I6YmVmb3JlLFxuLnJhaWYtc3RyZWFtaW5nLWN1cnNvcjphZnRlciB7XG4gIGRpc3BsYXk6IG5vbmU7XG59XG5cbkBrZXlmcmFtZXMgYmxpbmsge1xuICAwJSwgNTAlIHtcbiAgICBvcGFjaXR5OiAxO1xuICB9XG4gIDUxJSwgMTAwJSB7XG4gICAgb3BhY2l0eTogMDtcbiAgfVxufVxuQGtleWZyYW1lcyByb3RhdGUge1xuICAwJSB7XG4gICAgdHJhbnNmb3JtOiB0cmFuc2xhdGUoLTUwJSwgLTUwJSkgcm90YXRlWigwZGVnKTtcbiAgfVxuICAxMDAlIHtcbiAgICB0cmFuc2Zvcm06IHRyYW5zbGF0ZSgtNTAlLCAtNTAlKSByb3RhdGVaKDM2MGRlZyk7XG4gIH1cbn1cbkBrZXlmcmFtZXMgcm90YXRlY2N3IHtcbiAgMCUge1xuICAgIHRyYW5zZm9ybTogdHJhbnNsYXRlKC01MCUsIC01MCUpIHJvdGF0ZSgwZGVnKTtcbiAgfVxuICAxMDAlIHtcbiAgICB0cmFuc2Zvcm06IHRyYW5zbGF0ZSgtNTAlLCAtNTAlKSByb3RhdGUoLTM2MGRlZyk7XG4gIH1cbn1cbkBrZXlmcmFtZXMgc3BpbiB7XG4gIDAlLCAxMDAlIHtcbiAgICBib3gtc2hhZG93OiAwLjNlbSAwcHggMCAwcHggY3VycmVudGNvbG9yO1xuICB9XG4gIDEyJSB7XG4gICAgYm94LXNoYWRvdzogMC4zZW0gMC4zZW0gMCAwIGN1cnJlbnRjb2xvcjtcbiAgfVxuICAyNSUge1xuICAgIGJveC1zaGFkb3c6IDAgMC4zZW0gMCAwcHggY3VycmVudGNvbG9yO1xuICB9XG4gIDM3JSB7XG4gICAgYm94LXNoYWRvdzogLTAuM2VtIDAuM2VtIDAgMCBjdXJyZW50Y29sb3I7XG4gIH1cbiAgNTAlIHtcbiAgICBib3gtc2hhZG93OiAtMC4zZW0gMCAwIDAgY3VycmVudGNvbG9yO1xuICB9XG4gIDYyJSB7XG4gICAgYm94LXNoYWRvdzogLTAuM2VtIC0wLjNlbSAwIDAgY3VycmVudGNvbG9yO1xuICB9XG4gIDc1JSB7XG4gICAgYm94LXNoYWRvdzogMHB4IC0wLjNlbSAwIDAgY3VycmVudGNvbG9yO1xuICB9XG4gIDg3JSB7XG4gICAgYm94LXNoYWRvdzogMC4zZW0gLTAuM2VtIDAgMCBjdXJyZW50Y29sb3I7XG4gIH1cbn1cbiJdfQ== */

data/app/assets/stylesheets/raif/loader.scss

@@ -28,6 +28,33 @@
   animation-delay: .4s;
 }
 
+// Streaming cursor - a simple blinking cursor
+.raif-streaming-cursor {
+  display: inline-block;
+  width: 2px;
+  height: 1.1em;
+  margin-bottom: -2px;
+  background-color: currentColor;
+  animation: blink 1s infinite;
+  transform: none;
+  border-radius: 0;
+  position: relative;
+}
+
+.raif-streaming-cursor:before,
+.raif-streaming-cursor:after {
+  display: none;
+}
+
+@keyframes blink {
+  0%, 50% {
+    opacity: 1;
+  }
+  51%, 100% {
+    opacity: 0;
+  }
+}
+
 @keyframes rotate {
   0% {
     transform: translate(-50%, -50%) rotateZ(0deg);
@@ -49,7 +76,6 @@
 }
 
 @keyframes spin {
-
   0%,
   100% {
     box-shadow: .3em 0px 0 0px currentcolor;

data/app/models/raif/concerns/llm_response_parsing.rb

@@ -3,6 +3,8 @@
 module Raif::Concerns::LlmResponseParsing
   extend ActiveSupport::Concern
 
+  ASCII_CONTROL_CHARS = /[\x00-\x1f\x7f]/
+
   included do
     normalizes :raw_response, with: ->(text){ text&.strip }
 
@@ -35,32 +37,36 @@ module Raif::Concerns::LlmResponseParsing
   # If the response format is HTML, it will be sanitized via ActionController::Base.helpers.sanitize.
   #
   # @return [Object] The parsed response.
-  def parsed_response
+  def parsed_response(force_reparse: false)
     return if raw_response.blank?
+    return @parsed_response if @parsed_response.present? && !force_reparse
 
-    @parsed_response ||= if response_format_json?
-      json = raw_response.gsub("```json", "").gsub("```", "")
-      JSON.parse(json)
+    @parsed_response = if response_format_json?
+      parse_json_response
     elsif response_format_html?
-      html = raw_response.strip.gsub("```html", "").chomp("```")
-      clean_html_fragment(html)
+      parse_html_response
     else
       raw_response.strip
     end
   end
 
-  def clean_html_fragment(html)
-    fragment = Nokogiri::HTML.fragment(html)
+  def parse_json_response
+    json = raw_response.gsub(/#{ASCII_CONTROL_CHARS}|^```json|```$/, "").strip
 
-    fragment.traverse do |node|
-      if node.text? && node.text.strip.empty?
-        node.remove
-      end
-    end
+    raise JSON::ParserError, "Invalid JSON" if json.blank?
 
-    allowed_tags = self.class.allowed_tags || Rails::HTML5::SafeListSanitizer.allowed_tags
-    allowed_attributes = self.class.allowed_attributes || Rails::HTML5::SafeListSanitizer.allowed_attributes
+    JSON.parse(json)
+  end
 
-    ActionController::Base.helpers.sanitize(fragment.to_html, tags: allowed_tags, attributes: allowed_attributes).strip
+  def parse_html_response
+    html = raw_response.strip.gsub("```html", "").chomp("```")
+
+    html_with_converted_links = Raif::Utils::HtmlFragmentProcessor.convert_markdown_links_to_html(html)
+    Raif::Utils::HtmlFragmentProcessor.clean_html_fragment(
+      html_with_converted_links,
+      allowed_tags: allowed_tags,
+      allowed_attributes: allowed_attributes
+    )
   end
+
 end

data/app/models/raif/concerns/llms/anthropic/tool_formatting.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+module Raif::Concerns::Llms::Anthropic::ToolFormatting
+  extend ActiveSupport::Concern
+
+  def build_tools_parameter(model_completion)
+    tools = []
+
+    # If we're looking for a JSON response, add a tool to the request that the model can use to provide a JSON response
+    if model_completion.response_format_json? && model_completion.json_response_schema.present?
+      tools << {
+        name: "json_response",
+        description: "Generate a structured JSON response based on the provided schema.",
+        input_schema: model_completion.json_response_schema
+      }
+    end
+
+    # If we support native tool use and have tools available, add them to the request
+    if supports_native_tool_use? && model_completion.available_model_tools.any?
+      model_completion.available_model_tools_map.each do |_tool_name, tool|
+        tools << if tool.provider_managed?
+          format_provider_managed_tool(tool)
+        else
+          {
+            name: tool.tool_name,
+            description: tool.tool_description,
+            input_schema: tool.tool_arguments_schema
+          }
+        end
+      end
+    end
+
+    tools
+  end
+
+  def format_provider_managed_tool(tool)
+    validate_provider_managed_tool_support!(tool)
+
+    case tool.name
+    when "Raif::ModelTools::ProviderManaged::WebSearch"
+      {
+        type: "web_search_20250305",
+        name: "web_search",
+        max_uses: 5
+      }
+    when "Raif::ModelTools::ProviderManaged::CodeExecution"
+      {
+        type: "code_execution_20250522",
+        name: "code_execution"
+      }
+    else
+      raise Raif::Errors::UnsupportedFeatureError,
+        "Invalid provider-managed tool: #{tool.name} for #{key}"
+    end
+  end
+end

data/app/models/raif/concerns/llms/{bedrock_claude → bedrock}/message_formatting.rb

@@ -1,9 +1,9 @@
 # frozen_string_literal: true
 
-module Raif::Concerns::Llms::BedrockClaude::MessageFormatting
+module Raif::Concerns::Llms::Bedrock::MessageFormatting
   extend ActiveSupport::Concern
 
-  def format_string_message(content)
+  def format_string_message(content, role: nil)
     { "text" => content }
   end
 
@@ -13,7 +13,7 @@ module Raif::Concerns::Llms::BedrockClaude::MessageFormatting
     elsif image_input.source_type == :file_content
       # The AWS Bedrock SDK requires data sent as bytes (and doesn't support base64 like everyone else)
       # The ModelCompletion stores the messages as JSON though, so it can't be raw bytes (it will throw an encoding error).
-      # We store the image data as base64 and then it will get converted to bytes in Raif::Llms::BedrockClaude#perform_model_completion!
+      # We store the image data as base64 and then it will get converted to bytes in Raif::Llms::Bedrock#perform_model_completion!
       # before sending to AWS.
       {
         "image" => {
@@ -34,7 +34,7 @@ module Raif::Concerns::Llms::BedrockClaude::MessageFormatting
     elsif file_input.source_type == :file_content
       # The AWS Bedrock SDK requires data sent as bytes (and doesn't support base64 like everyone else)
       # The ModelCompletion stores the messages as JSON though, so it can't be raw bytes (it will throw an encoding error).
-      # We store the image data as base64 and then it will get converted to bytes in Raif::Llms::BedrockClaude#perform_model_completion!
+      # We store the image data as base64 and then it will get converted to bytes in Raif::Llms::Bedrock#perform_model_completion!
       # before sending to AWS.
       {
         "document" => {

data/app/models/raif/concerns/llms/bedrock/tool_formatting.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+module Raif::Concerns::Llms::Bedrock::ToolFormatting
+  extend ActiveSupport::Concern
+
+  def build_tools_parameter(model_completion)
+    tools = []
+
+    # If we're looking for a JSON response, add a tool to the request that the model can use to provide a JSON response
+    if model_completion.response_format_json? && model_completion.json_response_schema.present?
+      tools << {
+        name: "json_response",
+        description: "Generate a structured JSON response based on the provided schema.",
+        input_schema: { json: model_completion.json_response_schema }
+      }
+    end
+
+    model_completion.available_model_tools_map.each do |_tool_name, tool|
+      tools << if tool.provider_managed?
+        raise Raif::Errors::UnsupportedFeatureError,
+          "Invalid provider-managed tool: #{tool.name} for #{key}"
+      else
+        {
+          name: tool.tool_name,
+          description: tool.tool_description,
+          input_schema: { json: tool.tool_arguments_schema }
+        }
+      end
+    end
+
+    return {} if tools.blank?
+
+    {
+      tools: tools.map{|tool| { tool_spec: tool } }
+    }
+  end
+end

data/app/models/raif/concerns/llms/message_formatting.rb

@@ -5,9 +5,10 @@ module Raif::Concerns::Llms::MessageFormatting
 
   def format_messages(messages)
     messages.map do |message|
+      role = message["role"] || message[:role]
       {
-        "role" => message["role"] || message[:role],
-        "content" => format_message_content(message["content"] || message[:content])
+        "role" => role,
+        "content" => format_message_content(message["content"] || message[:content], role: role)
       }
     end
   end
@@ -15,11 +16,11 @@ module Raif::Concerns::Llms::MessageFormatting
   # Content could be a string or an array.
   # If it's an array, it could contain Raif::ModelImageInput or Raif::ModelFileInput objects,
   # which need to be formatted according to each model provider's API.
-  def format_message_content(content)
+  def format_message_content(content, role: nil)
     raise ArgumentError,
       "Message content must be an array or a string. Content was: #{content.inspect}" unless content.is_a?(Array) || content.is_a?(String)
 
-    return [format_string_message(content)] if content.is_a?(String)
+    return [format_string_message(content, role: role)] if content.is_a?(String)
 
     content.map do |item|
       if item.is_a?(Raif::ModelImageInput)
@@ -27,14 +28,14 @@ module Raif::Concerns::Llms::MessageFormatting
       elsif item.is_a?(Raif::ModelFileInput)
         format_model_file_input_message(item)
       elsif item.is_a?(String)
-        format_string_message(item)
+        format_string_message(item, role: role)
       else
         item
       end
     end
   end
 
-  def format_string_message(content)
+  def format_string_message(content, role: nil)
     { "type" => "text", "text" => content }
   end