net-llm 0.3.1 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: de6c591968c95eafb5af04842bc5fc8f4f0235369a6f7010843ab9adbe2199cd
-  data.tar.gz: f8dbd38c119eacc38814a5eb173f1a2158564bce75dd5c6d111d3900e17f8136
+  metadata.gz: a7168d17a456b69a77ada9a8c5eb855ae6730a124cdd55dd9e72fee2bfa6fef1
+  data.tar.gz: 0fde66f7b0304486f3c5da851d7bbe97347f58fa0692dbd48c8462932ebe1bda
 SHA512:
-  metadata.gz: dd8e9c389c9ed5f0614b386cf56a6359275d156197b4fe284671fc0f062d839181b938f91b898b472cf71f2d9090f5aedb9b4b725616ba97acf9f072f1a5a2a3
-  data.tar.gz: 3ac65c38e85ff03fd29e1a9dd227d674455773a35290b6871ac18d05f844afc6142a0daebcb2c88a75ec858d57b0fc7c9eddb5b07a2e4a1c10917cb6156be0bd
+  metadata.gz: 1969cb1afc0e322f9899f97c95898c9ebf825fd826ce43af0fc0dc66fe86b298ca4538d3426519791661824265d6ff0ce51f3ecd2012b464639a969df4c1142f
+  data.tar.gz: 2d765715615d2d66f36fd00e8a64360061f740c8e19af1d1ef80c9d5d1b744964e5f3e741bc7af414d877acec7840adf855ba28159131bad31bcdebbe99dd5b5

data/CHANGELOG.md

@@ -1,5 +1,47 @@
 ## [Unreleased]
 
+## [0.5.0] - 2025-01-07
+
+### Added
+- VertexAI provider for Claude models via Google Cloud
+  - Uses Application Default Credentials (ADC) for authentication
+  - Supports streaming and non-streaming modes
+  - Model routing with NotImplementedError for unsupported models
+- Unified `fetch(messages, tools = [], &block)` method across all providers
+  - Normalized response format with `:delta` and `:complete` types
+  - Consistent `tool_calls` structure: `{ id:, name:, arguments: }`
+  - Thinking content support in streaming responses
+- Claude class for shared Anthropic protocol logic
+  - Automatic system message extraction from messages array
+  - Message normalization for tool results and tool_calls
+- Environment variable support for provider configuration
+  - `OLLAMA_HOST` for Ollama (default: localhost:11434)
+  - `OPENAI_API_KEY` and `OPENAI_BASE_URL` for OpenAI
+  - `ANTHROPIC_API_KEY` for Anthropic
+  - `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_REGION` for VertexAI
+
+### Changed
+- Refactored Anthropic provider to delegate to Claude class
+- Refactored VertexAI provider to delegate to Claude class
+- Updated default Anthropic model to claude-sonnet-4-20250514
+- Updated default VertexAI model to claude-opus-4-5@20251101
+
+### Fixed
+- Fixed streaming tool_calls accumulation in Ollama provider
+- Fixed error responses to include response body for debugging
+- Fixed VertexAI model name format (@ separator instead of -)
+
+## [0.4.0] - 2025-10-15
+### Added
+- Added tool/function calling support to Ollama provider
+- Ollama `chat` method now accepts optional `tools` parameter matching OpenAI signature
+- Tools work in both streaming and non-streaming modes
+- Added comprehensive test coverage for tool functionality
+
+### Changed
+- Updated README with Ollama tools example
+- Updated API coverage documentation
+
 ## [0.3.1] - 2025-10-08
 ### Fixed
 - Added missing net-hippie runtime dependency to gemspec

data/README.md

@@ -1,6 +1,6 @@
 # Net::Llm
 
-A minimal Ruby gem providing interfaces to connect to OpenAI, Ollama, and Anthropic (Claude) LLM APIs.
+A minimal Ruby gem providing interfaces to connect to OpenAI, Ollama, Anthropic (Claude), and VertexAI LLM APIs.
 
 ## Installation
 
@@ -87,6 +87,29 @@ response = client.chat(messages)
 puts response['message']['content']
 ```
 
+#### With Tools
+
+```ruby
+tools = [
+  {
+    type: 'function',
+    function: {
+      name: 'get_weather',
+      description: 'Get current weather',
+      parameters: {
+        type: 'object',
+        properties: {
+          location: { type: 'string' }
+        },
+        required: ['location']
+      }
+    }
+  }
+]
+
+response = client.chat(messages, tools)
+```
+
 #### Streaming
 
 ```ruby
@@ -121,7 +144,7 @@ require 'net/llm'
 
 client = Net::Llm::Anthropic.new(
   api_key: ENV['ANTHROPIC_API_KEY'],
-  model: 'claude-3-5-sonnet-20241022'
+  model: 'claude-sonnet-4-20250514'
 )
 
 messages = [
@@ -171,6 +194,54 @@ tools = [
 response = client.messages(messages, tools: tools)
 ```
 
+### VertexAI
+
+```ruby
+require 'net/llm'
+
+client = Net::Llm::VertexAI.new(
+  project_id: ENV['GOOGLE_CLOUD_PROJECT'],
+  region: ENV.fetch('GOOGLE_CLOUD_REGION', 'us-east5'),
+  model: 'claude-opus-4-5@20251101'
+)
+
+messages = [
+  { role: 'user', content: 'Hello!' }
+]
+
+response = client.messages(messages)
+puts response.dig('content', 0, 'text')
+```
+
+Uses Application Default Credentials (ADC) for authentication. Run `gcloud auth application-default login` to configure.
+
+### Unified Fetch Interface
+
+All providers support a unified `fetch` method with a normalized response format:
+
+```ruby
+result = client.fetch(messages, tools)
+
+result[:type]        # :complete
+result[:content]     # "Response text"
+result[:thinking]    # Extended thinking (Claude only)
+result[:tool_calls]  # [{ id:, name:, arguments: }]
+result[:stop_reason] # :end_turn, :tool_use, :max_tokens
+```
+
+#### Streaming
+
+```ruby
+client.fetch(messages, tools) do |chunk|
+  case chunk[:type]
+  when :delta
+    print chunk[:content]
+  when :complete
+    puts "\nDone: #{chunk[:stop_reason]}"
+  end
+end
+```
+
 ## Error Handling
 
 All non-streaming API methods return error information as a hash when requests fail:
@@ -195,7 +266,7 @@ Streaming methods still raise exceptions on HTTP errors.
 - `/v1/embeddings`
 
 ### Ollama
-- `/api/chat` (with streaming)
+- `/api/chat` (with streaming and tools)
 - `/api/generate` (with streaming)
 - `/api/embed`
 - `/api/tags`
@@ -204,6 +275,9 @@ Streaming methods still raise exceptions on HTTP errors.
 ### Anthropic (Claude)
 - `/v1/messages` (with streaming and tools)
 
+### VertexAI
+- Claude models via Google Cloud AI Platform (with streaming and tools)
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
@@ -212,7 +286,7 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/xlgmokha/net-llm.
+Send me an email. For instructions see https://git-send-email.io/.
 
 ## License
 

data/lib/net/llm/anthropic.rb

@@ -3,99 +3,21 @@
 module Net
   module Llm
     class Anthropic
-      attr_reader :api_key, :model, :http
+      attr_reader :api_key, :model
 
-      def initialize(api_key:, model: "claude-3-5-sonnet-20241022", http: Net::Llm.http)
+      def initialize(api_key: ENV.fetch("ANTHROPIC_API_KEY"), model: "claude-sonnet-4-20250514", http: Net::Llm.http)
         @api_key = api_key
         @model = model
-        @http = http
-      end
-
-      def messages(messages, system: nil, max_tokens: 1024, tools: nil, &block)
-        url = "https://api.anthropic.com/v1/messages"
-        payload = build_payload(messages, system, max_tokens, tools, block_given?)
-
-        if block_given?
-          stream_request(url, payload, &block)
-        else
-          post_request(url, payload)
-        end
-      end
-
-      private
-
-      def build_payload(messages, system, max_tokens, tools, stream)
-        payload = {
+        @claude = Claude.new(
+          endpoint: "https://api.anthropic.com/v1/messages",
+          headers: { "x-api-key" => api_key, "anthropic-version" => "2023-06-01" },
           model: model,
-          max_tokens: max_tokens,
-          messages: messages,
-          stream: stream
-        }
-        payload[:system] = system if system
-        payload[:tools] = tools if tools
-        payload
-      end
-
-      def headers
-        {
-          "x-api-key" => api_key,
-          "anthropic-version" => "2023-06-01"
-        }
-      end
-
-      def post_request(url, payload)
-        handle_response(http.post(url, headers: headers, body: payload))
-      end
-
-      def handle_response(response)
-        if response.is_a?(Net::HTTPSuccess)
-          JSON.parse(response.body)
-        else
-          { "code" => response.code, "body" => response.body }
-        end
+          http: http
+        )
       end
 
-      def stream_request(url, payload, &block)
-        http.post(url, headers: headers, body: payload) do |response|
-          raise "HTTP #{response.code}" unless response.is_a?(Net::HTTPSuccess)
-
-          buffer = ""
-          response.read_body do |chunk|
-            buffer += chunk
-
-            while (event = extract_sse_event(buffer))
-              next if event[:data].nil? || event[:data].empty?
-              next if event[:data] == "[DONE]"
-
-              json = JSON.parse(event[:data])
-              block.call(json)
-
-              break if json["type"] == "message_stop"
-            end
-          end
-        end
-      end
-
-      def extract_sse_event(buffer)
-        event_end = buffer.index("\n\n")
-        return nil unless event_end
-
-        event_data = buffer[0...event_end]
-        buffer.replace(buffer[(event_end + 2)..-1] || "")
-
-        event = {}
-        event_data.split("\n").each do |line|
-          if line.start_with?("event: ")
-            event[:event] = line[7..-1]
-          elsif line.start_with?("data: ")
-            event[:data] = line[6..-1]
-          elsif line == "data:"
-            event[:data] = ""
-          end
-        end
-
-        event
-      end
+      def messages(...) = @claude.messages(...)
+      def fetch(...) = @claude.fetch(...)
     end
   end
 end

data/lib/net/llm/claude.rb

@@ -0,0 +1,266 @@
+# frozen_string_literal: true
+
+module Net
+  module Llm
+    class Claude
+      attr_reader :endpoint, :headers, :model, :http, :anthropic_version
+
+      def initialize(endpoint:, headers:, http:, model: nil, anthropic_version: nil)
+        @endpoint = endpoint
+        @headers_source = headers
+        @model = model
+        @http = http
+        @anthropic_version = anthropic_version
+      end
+
+      def headers
+        @headers_source.respond_to?(:call) ? @headers_source.call : @headers_source
+      end
+
+      def messages(messages, system: nil, max_tokens: 1024, tools: nil, &block)
+        payload = build_payload(messages, system, max_tokens, tools, block_given?)
+
+        if block_given?
+          stream_request(payload, &block)
+        else
+          post_request(payload)
+        end
+      end
+
+      def fetch(messages, tools = [], &block)
+        system_message, user_messages = extract_system_message(messages)
+        anthropic_tools = tools.empty? ? nil : tools.map { |t| normalize_tool_for_anthropic(t) }
+
+        if block_given?
+          fetch_streaming(user_messages, anthropic_tools, system: system_message, &block)
+        else
+          fetch_non_streaming(user_messages, anthropic_tools, system: system_message)
+        end
+      end
+
+      private
+
+      def build_payload(messages, system, max_tokens, tools, stream)
+        payload = { max_tokens: max_tokens, messages: messages, stream: stream }
+        payload[:model] = model if model
+        payload[:anthropic_version] = anthropic_version if anthropic_version
+        payload[:system] = system if system
+        payload[:tools] = tools if tools
+        payload
+      end
+
+      def post_request(payload)
+        handle_response(http.post(endpoint, headers: headers, body: payload))
+      end
+
+      def handle_response(response)
+        if response.is_a?(Net::HTTPSuccess)
+          JSON.parse(response.body)
+        else
+          { "code" => response.code, "body" => response.body }
+        end
+      end
+
+      def stream_request(payload, &block)
+        http.post(endpoint, headers: headers, body: payload) do |response|
+          raise "HTTP #{response.code}: #{response.body}" unless response.is_a?(Net::HTTPSuccess)
+
+          buffer = ""
+          response.read_body do |chunk|
+            buffer += chunk
+
+            while (event = extract_sse_event(buffer))
+              next if event[:data].nil? || event[:data].empty?
+              next if event[:data] == "[DONE]"
+
+              json = JSON.parse(event[:data])
+              block.call(json)
+
+              break if json["type"] == "message_stop"
+            end
+          end
+        end
+      end
+
+      def extract_sse_event(buffer)
+        event_end = buffer.index("\n\n")
+        return nil unless event_end
+
+        event_data = buffer[0...event_end]
+        buffer.replace(buffer[(event_end + 2)..] || "")
+
+        event = {}
+        event_data.split("\n").each do |line|
+          if line.start_with?("event: ")
+            event[:event] = line[7..]
+          elsif line.start_with?("data: ")
+            event[:data] = line[6..]
+          elsif line == "data:"
+            event[:data] = ""
+          end
+        end
+
+        event
+      end
+
+      def extract_system_message(messages)
+        system_msg = messages.find { |m| m[:role] == "system" || m["role"] == "system" }
+        system_content = system_msg ? (system_msg[:content] || system_msg["content"]) : nil
+        other_messages = messages.reject { |m| m[:role] == "system" || m["role"] == "system" }
+        normalized_messages = normalize_messages_for_claude(other_messages)
+        [system_content, normalized_messages]
+      end
+
+      def normalize_messages_for_claude(messages)
+        messages.map do |msg|
+          role = msg[:role] || msg["role"]
+          tool_calls = msg[:tool_calls] || msg["tool_calls"]
+
+          if role == "tool"
+            {
+              role: "user",
+              content: [{
+                type: "tool_result",
+                tool_use_id: msg[:tool_call_id] || msg["tool_call_id"],
+                content: msg[:content] || msg["content"]
+              }]
+            }
+          elsif role == "assistant" && tool_calls&.any?
+            content = []
+            text = msg[:content] || msg["content"]
+            content << { type: "text", text: text } if text && !text.empty?
+            tool_calls.each do |tc|
+              func = tc[:function] || tc["function"] || {}
+              args = func[:arguments] || func["arguments"]
+              input = args.is_a?(String) ? (JSON.parse(args) rescue {}) : (args || {})
+              content << {
+                type: "tool_use",
+                id: tc[:id] || tc["id"],
+                name: func[:name] || func["name"] || tc[:name] || tc["name"],
+                input: input
+              }
+            end
+            { role: "assistant", content: content }
+          else
+            msg
+          end
+        end
+      end
+
+      def fetch_non_streaming(messages, tools, system: nil)
+        result = self.messages(messages, system: system, tools: tools)
+        return result if result["code"]
+
+        {
+          type: :complete,
+          content: extract_text_content(result["content"]),
+          thinking: extract_thinking_content(result["content"]),
+          tool_calls: extract_tool_calls(result["content"]),
+          stop_reason: map_stop_reason(result["stop_reason"])
+        }
+      end
+
+      def fetch_streaming(messages, tools, system: nil, &block)
+        content = ""
+        thinking = ""
+        tool_calls = []
+        stop_reason = :end_turn
+
+        self.messages(messages, system: system, tools: tools) do |event|
+          case event["type"]
+          when "content_block_start"
+            if event.dig("content_block", "type") == "tool_use"
+              tool_calls << {
+                id: event.dig("content_block", "id"),
+                name: event.dig("content_block", "name"),
+                arguments: {}
+              }
+            end
+          when "content_block_delta"
+            delta = event["delta"]
+            case delta["type"]
+            when "text_delta"
+              text = delta["text"]
+              content += text
+              block.call({ type: :delta, content: text, thinking: nil, tool_calls: nil })
+            when "thinking_delta"
+              text = delta["thinking"]
+              thinking += text if text
+              block.call({ type: :delta, content: nil, thinking: text, tool_calls: nil })
+            when "input_json_delta"
+              if tool_calls.any?
+                tool_calls.last[:arguments_json] ||= ""
+                tool_calls.last[:arguments_json] += delta["partial_json"] || ""
+              end
+            end
+          when "message_delta"
+            stop_reason = map_stop_reason(event.dig("delta", "stop_reason"))
+          when "message_stop"
+            tool_calls.each do |tc|
+              if tc[:arguments_json]
+                tc[:arguments] = begin
+                  JSON.parse(tc[:arguments_json])
+                rescue
+                  {}
+                end
+                tc.delete(:arguments_json)
+              end
+            end
+            block.call({
+              type: :complete,
+              content: content,
+              thinking: thinking.empty? ? nil : thinking,
+              tool_calls: tool_calls,
+              stop_reason: stop_reason
+            })
+          end
+        end
+      end
+
+      def extract_text_content(content_blocks)
+        return nil unless content_blocks
+
+        content_blocks
+          .select { |b| b["type"] == "text" }
+          .map { |b| b["text"] }
+          .join
+      end
+
+      def extract_thinking_content(content_blocks)
+        return nil unless content_blocks
+
+        thinking = content_blocks
+          .select { |b| b["type"] == "thinking" }
+          .map { |b| b["thinking"] }
+          .join
+
+        thinking.empty? ? nil : thinking
+      end
+
+      def extract_tool_calls(content_blocks)
+        return [] unless content_blocks
+
+        content_blocks
+          .select { |b| b["type"] == "tool_use" }
+          .map { |b| { id: b["id"], name: b["name"], arguments: b["input"] || {} } }
+      end
+
+      def normalize_tool_for_anthropic(tool)
+        if tool[:function]
+          { name: tool[:function][:name], description: tool[:function][:description], input_schema: tool[:function][:parameters] }
+        else
+          tool
+        end
+      end
+
+      def map_stop_reason(reason)
+        case reason
+        when "end_turn" then :end_turn
+        when "tool_use" then :tool_use
+        when "max_tokens" then :max_tokens
+        else :end_turn
+        end
+      end
+    end
+  end
+end

data/lib/net/llm/ollama.rb

@@ -5,75 +5,121 @@ module Net
     class Ollama
       attr_reader :host, :model, :http
 
-      def initialize(host: "localhost:11434", model: "llama2", http: Net::Llm.http)
+      def initialize(host: ENV.fetch("OLLAMA_HOST", "localhost:11434"), model: "gpt-oss", http: Net::Llm.http)
         @host = host
         @model = model
         @http = http
       end
 
-      def chat(messages, &block)
-        url = build_url("/api/chat")
+      def chat(messages, tools = [], &block)
         payload = { model: model, messages: messages, stream: block_given? }
+        payload[:tools] = tools unless tools.empty?
 
-        if block_given?
-          stream_request(url, payload, &block)
-        else
-          post_request(url, payload)
-        end
+        execute(build_url("/api/chat"), payload, &block)
       end
 
-      def generate(prompt, &block)
-        url = build_url("/api/generate")
-        payload = { model: model, prompt: prompt, stream: block_given? }
+      def fetch(messages, tools = [], &block)
+        content = ""
+        thinking = ""
+        tool_calls = []
 
         if block_given?
-          stream_request(url, payload, &block)
+          chat(messages, tools) do |chunk|
+            msg = chunk["message"] || {}
+            delta_content = msg["content"]
+            delta_thinking = msg["thinking"]
+
+            content += delta_content if delta_content
+            thinking += delta_thinking if delta_thinking
+            tool_calls += normalize_tool_calls(msg["tool_calls"]) if msg["tool_calls"]
+
+            if chunk["done"]
+              block.call({
+                type: :complete,
+                content: content,
+                thinking: thinking.empty? ? nil : thinking,
+                tool_calls: tool_calls,
+                stop_reason: map_stop_reason(chunk["done_reason"])
+              })
+            else
+              block.call({
+                type: :delta,
+                content: delta_content,
+                thinking: delta_thinking,
+                tool_calls: nil
+              })
+            end
+          end
         else
-          post_request(url, payload)
+          result = chat(messages, tools)
+          msg = result["message"] || {}
+          {
+            type: :complete,
+            content: msg["content"],
+            thinking: msg["thinking"],
+            tool_calls: normalize_tool_calls(msg["tool_calls"]),
+            stop_reason: map_stop_reason(result["done_reason"])
+          }
         end
       end
 
+      def generate(prompt, &block)
+        execute(build_url("/api/generate"), {
+          model: model,
+          prompt: prompt,
+          stream: block_given?
+        }, &block)
+      end
+
       def embeddings(input)
-        url = build_url("/api/embed")
-        payload = { model: model, input: input }
-        post_request(url, payload)
+        post_request(build_url("/api/embed"), { model: model, input: input })
       end
 
       def tags
-        url = build_url("/api/tags")
-        response = http.get(url)
-        handle_response(response)
+        get_request(build_url("/api/tags"))
       end
 
       def show(name)
-        url = build_url("/api/show")
-        payload = { name: name }
-        post_request(url, payload)
+        post_request(build_url("/api/show"), { name: name })
       end
 
       private
 
+      def execute(url, payload, &block)
+        if block_given?
+          stream_request(url, payload, &block)
+        else
+          post_request(url, payload)
+        end
+      end
+
       def build_url(path)
         base = host.start_with?("http://", "https://") ? host : "http://#{host}"
         "#{base}#{path}"
       end
 
+      def get_request(url)
+        handle_response(http.get(url))
+      end
+
       def post_request(url, payload)
-        response = http.post(url, body: payload)
-        handle_response(response)
+        handle_response(http.post(url, body: payload))
       end
 
       def handle_response(response)
         if response.is_a?(Net::HTTPSuccess)
           JSON.parse(response.body)
         else
-          { "code" => response.code, "body" => response.body }
+          {
+            "code" => response.code,
+            "body" => response.body
+          }
         end
       end
 
       def stream_request(url, payload, &block)
         http.post(url, body: payload) do |response|
-          raise "HTTP #{response.code}" unless response.is_a?(Net::HTTPSuccess)
+          raise "HTTP #{response.code}: #{response.body}" unless response.is_a?(Net::HTTPSuccess)
 
           buffer = ""
           response.read_body do |chunk|
@@ -99,6 +145,27 @@ module Net
         buffer.replace(buffer[(message_end + 1)..-1] || "")
         message
       end
+
+      def normalize_tool_calls(tool_calls)
+        return [] if tool_calls.nil? || tool_calls.empty?
+
+        tool_calls.map do |tc|
+          {
+            id: tc["id"] || tc.dig("function", "id"),
+            name: tc.dig("function", "name"),
+            arguments: tc.dig("function", "arguments") || {}
+          }
+        end
+      end
+
+      def map_stop_reason(reason)
+        case reason
+        when "stop" then :end_turn
+        when "tool_calls", "tool_use" then :tool_use
+        when "length" then :max_tokens
+        else :end_turn
+        end
+      end
     end
   end
 end

data/lib/net/llm/openai.rb

@@ -5,7 +5,7 @@ module Net
     class OpenAI
       attr_reader :api_key, :base_url, :model, :http
 
-      def initialize(api_key:, base_url: "https://api.openai.com/v1", model: "gpt-4o-mini", http: Net::Llm.http)
+      def initialize(api_key: ENV.fetch("OPENAI_API_KEY"), base_url: ENV.fetch("OPENAI_BASE_URL", "https://api.openai.com/v1"), model: "gpt-4o-mini", http: Net::Llm.http)
         @api_key = api_key
         @base_url = base_url
         @model = model
@@ -20,6 +20,14 @@ module Net
         ))
       end
 
+      def fetch(messages, tools = [], &block)
+        if block_given?
+          fetch_streaming(messages, tools, &block)
+        else
+          fetch_non_streaming(messages, tools)
+        end
+      end
+
       def models
         handle_response(http.get("#{base_url}/models", headers: headers))
       end
@@ -45,6 +53,120 @@ module Net
           { "code" => response.code, "body" => response.body }
         end
       end
+
+      def fetch_non_streaming(messages, tools)
+        body = { model: model, messages: messages }
+        body[:tools] = tools unless tools.empty?
+        body[:tool_choice] = "auto" unless tools.empty?
+
+        result = handle_response(http.post("#{base_url}/chat/completions", headers: headers, body: body))
+        return result if result["code"]
+
+        msg = result.dig("choices", 0, "message") || {}
+        {
+          type: :complete,
+          content: msg["content"],
+          thinking: nil,
+          tool_calls: normalize_tool_calls(msg["tool_calls"]),
+          stop_reason: map_stop_reason(result.dig("choices", 0, "finish_reason"))
+        }
+      end
+
+      def fetch_streaming(messages, tools, &block)
+        body = { model: model, messages: messages, stream: true }
+        body[:tools] = tools unless tools.empty?
+        body[:tool_choice] = "auto" unless tools.empty?
+
+        content = ""
+        tool_calls = {}
+        stop_reason = :end_turn
+
+        http.post("#{base_url}/chat/completions", headers: headers, body: body) do |response|
+          raise "HTTP #{response.code}: #{response.body}" unless response.is_a?(Net::HTTPSuccess)
+
+          buffer = ""
+          response.read_body do |chunk|
+            buffer += chunk
+
+            while (line = extract_line(buffer))
+              next if line.empty? || !line.start_with?("data: ")
+
+              data = line[6..]
+              break if data == "[DONE]"
+
+              json = JSON.parse(data)
+              delta = json.dig("choices", 0, "delta") || {}
+
+              if delta["content"]
+                content += delta["content"]
+                block.call({ type: :delta, content: delta["content"], thinking: nil, tool_calls: nil })
+              end
+
+              if delta["tool_calls"]
+                delta["tool_calls"].each do |tc|
+                  idx = tc["index"]
+                  tool_calls[idx] ||= { id: nil, name: nil, arguments_json: "" }
+                  tool_calls[idx][:id] = tc["id"] if tc["id"]
+                  tool_calls[idx][:name] = tc.dig("function", "name") if tc.dig("function", "name")
+                  tool_calls[idx][:arguments_json] += tc.dig("function", "arguments") || ""
+                end
+              end
+
+              if json.dig("choices", 0, "finish_reason")
+                stop_reason = map_stop_reason(json.dig("choices", 0, "finish_reason"))
+              end
+            end
+          end
+        end
+
+        final_tool_calls = tool_calls.values.map do |tc|
+          args = begin
+            JSON.parse(tc[:arguments_json])
+          rescue
+            {}
+          end
+          { id: tc[:id], name: tc[:name], arguments: args }
+        end
+
+        block.call({
+          type: :complete,
+          content: content,
+          thinking: nil,
+          tool_calls: final_tool_calls,
+          stop_reason: stop_reason
+        })
+      end
+
+      def extract_line(buffer)
+        line_end = buffer.index("\n")
+        return nil unless line_end
+
+        line = buffer[0...line_end]
+        buffer.replace(buffer[(line_end + 1)..] || "")
+        line
+      end
+
+      def normalize_tool_calls(tool_calls)
+        return [] if tool_calls.nil? || tool_calls.empty?
+
+        tool_calls.map do |tc|
+          args = tc.dig("function", "arguments")
+          {
+            id: tc["id"],
+            name: tc.dig("function", "name"),
+            arguments: args.is_a?(String) ? (JSON.parse(args) rescue {}) : (args || {})
+          }
+        end
+      end
+
+      def map_stop_reason(reason)
+        case reason
+        when "stop" then :end_turn
+        when "tool_calls" then :tool_use
+        when "length" then :max_tokens
+        else :end_turn
+        end
+      end
     end
   end
 end

data/lib/net/llm/version.rb

@@ -2,6 +2,6 @@
 
 module Net
   module Llm
-    VERSION = "0.3.1"
+    VERSION = "0.5.0"
   end
 end

data/lib/net/llm/vertex_ai.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+module Net
+  module Llm
+    class VertexAI
+      attr_reader :project_id, :region, :model
+
+      def initialize(project_id: ENV.fetch("GOOGLE_CLOUD_PROJECT"), region: ENV.fetch("GOOGLE_CLOUD_REGION", "us-east5"), model: "claude-opus-4-5@20251101", http: Net::Llm.http)
+        @project_id = project_id
+        @region = region
+        @model = model
+        @handler = build_handler(http)
+      end
+
+      def messages(...) = @handler.messages(...)
+      def fetch(...) = @handler.fetch(...)
+
+      private
+
+      def build_handler(http)
+        if model.start_with?("claude-")
+          Claude.new(
+            endpoint: "https://#{region}-aiplatform.googleapis.com/v1/projects/#{project_id}/locations/#{region}/publishers/anthropic/models/#{model}:rawPredict",
+            headers: -> { { "Authorization" => "Bearer #{access_token}" } },
+            http: http,
+            anthropic_version: "vertex-2023-10-16"
+          )
+        else
+          raise NotImplementedError, "Model '#{model}' is not yet supported. Only Claude models (claude-*) are currently implemented."
+        end
+      end
+
+      def access_token
+        @access_token ||= `gcloud auth application-default print-access-token`.strip
+      end
+    end
+  end
+end

data/lib/net/llm.rb

@@ -3,7 +3,9 @@
 require_relative "llm/version"
 require_relative "llm/openai"
 require_relative "llm/ollama"
+require_relative "llm/claude"
 require_relative "llm/anthropic"
+require_relative "llm/vertex_ai"
 require "net/hippie"
 require "json"
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: net-llm
 version: !ruby/object:Gem::Version
-  version: 0.3.1
+  version: 0.5.0
 platform: ruby
 authors:
 - mo khan
@@ -51,6 +51,62 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: vcr
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
 description: A minimal Ruby gem providing interfaces to connect to OpenAI, Ollama,
   and Anthropic (Claude) LLM APIs
 email:
@@ -65,17 +121,19 @@ files:
 - Rakefile
 - lib/net/llm.rb
 - lib/net/llm/anthropic.rb
+- lib/net/llm/claude.rb
 - lib/net/llm/ollama.rb
 - lib/net/llm/openai.rb
 - lib/net/llm/version.rb
+- lib/net/llm/vertex_ai.rb
 - sig/net/llm.rbs
-homepage: https://github.com/xlgmokha/net-llm
+homepage: https://src.mokhan.ca/xlgmokha/net-llm/
 licenses:
 - MIT
 metadata:
-  homepage_uri: https://github.com/xlgmokha/net-llm
-  source_code_uri: https://github.com/xlgmokha/net-llm
-  changelog_uri: https://github.com/xlgmokha/net-llm/blob/main/CHANGELOG.md
+  homepage_uri: https://src.mokhan.ca/xlgmokha/net-llm/
+  source_code_uri: https://src.mokhan.ca/xlgmokha/net-llm/
+  changelog_uri: https://src.mokhan.ca/xlgmokha/net-llm/blob/main/CHANGELOG.md.html
 rdoc_options: []
 require_paths:
 - lib