mistral 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 01bc1534430ff8a86dafca215d434c999337163910c2b67c5a7198bbc4ade7c3
-  data.tar.gz: d97eda8d0b53118de9998dca31041ec14a1e357b7052a6083d4f172408192ebf
+  metadata.gz: a13548c7603e4a353f94fb7c798f3256c8c84fcb24f6b9a35a0d34539026641a
+  data.tar.gz: 12d4ebe2515e9970d00e086c90bd69c10ab86d7e786b8417d534b3d6b30fc091
 SHA512:
-  metadata.gz: 46f9cb65643ff7b805a2fe94181d7d8af3e27afc42ae001d9a6a6a4ef660ae79461508c793734cf46bcf14e470ec2f2e21c480e12d567f2e0c224c6cd3639f62
-  data.tar.gz: a68462d1daf574b786d325ece5f5519d9b3bada3081b1a150eb088e6bdb036b6f455a46a5e516ed6b4f6a79b294e400c21240c0a210181279c5eab16575756e9
+  metadata.gz: 29ce8bafc147e843e8d9f8057a9b6fde4ab331ae1bd39d5b19e73fd2cc362940368d84c97c3433b69d80ec516ad710e0314e2eab509e1706cc4b206e77787ad7
+  data.tar.gz: 93013c03b1047734c3a97b3a20bc98b5d3768bc2d2b0ac64a67a4b5d14bb03c85d16fc146b8eb8d87690278f9012eccfe4ca07d67357d1b66310ce19a36fb1e2

data/.env.example

@@ -1,3 +1,3 @@
 # Get this from https://console.mistral.ai/api-keys/
 MISTRAL_API_KEY=
-LOG_LEVEL=ERROR
+MISTRAL_LOG_LEVEL=ERROR

data/CHANGELOG.md

@@ -5,8 +5,35 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.1.1/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+## [0.3.0] - 2024-05-30
+
+### Added
+
+- Added support for completion requests which you can use to query their latest model
+[codestral](https://mistral.ai/news/codestral/).
+See [this example](https://github.com/wilsonsilva/mistral/blob/0.3.0/examples/code_completion.rb) to get started.
+
+## [0.2.0] - 2024-05-23
+
+### Added
+
+- We now support tool_call_id for tool messages. This will be mandatory in the future but you can start using right
+away to improve the model performance during function calling (especially multiple).
+Ports [mistralai/client-python#93](https://github.com/mistralai/client-python/pull/93)
+
+### Changed
+
+- Renamed `LOG_LEVEL` to `MISTRAL_LOG_LEVEL`. This is not a direct port of the Python client because Python has a
+global logger in the `logging` module, but Ruby doesn't.
+Ports [mistralai/client-python#86](https://github.com/mistralai/client-python/pull/86)
+- Get API key at client initialization. Ports
+[mistralai/client-python#57](https://github.com/mistralai/client-python/pull/57)
+
 ## [0.1.0] - 2024-05-04
 
-- Initial release. Feature parity with `v0.1.8` of the [client-python](https://github.com/mistralai/client-python)
+- Initial release. Feature parity with `v0.1.8` of the
+[mistralai/client-python](https://github.com/mistralai/client-python)
 
-[0.1.0]: https://github.com/wilsonsilva/nostr/compare/28e7c9...v0.1.0
+[0.3.0]: https://github.com/wilsonsilva/mistral/compare/v0.2.0...v0.3.0
+[0.2.0]: https://github.com/wilsonsilva/mistral/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/wilsonsilva/mistral/compare/28e7c9...v0.1.0

data/PYTHON_CLIENT_COMPARISON.md

@@ -175,10 +175,21 @@ This code resides in `lib/http/features/line_iterable_body.rb`.
 ## Testing
 
 The Ruby gem aims for 1:1 parity with the Python client. As such, it uses `Minitest` (similar to Python's `pytest`).
-However, testing was simplified by using and `webmock` for stubbing requests, instead of implementing 100% test
+However, testing was simplified by using `webmock` for stubbing requests, instead of implementing 100% test
 coverage and using RSpec, which is usually what I do.
 
 ## Examples
 
 The `function_calling.rb` example omits the unnecessary `n_rows = data['transaction_id'].length` line present in
 the Python version.
+
+## Logging
+
+Python has a global logger:
+
+```python
+self._logger = logging.getLogger(__name__)
+```
+
+Ruby doesn't. Thus, in order to allow users to customize the logging level, they can set the environment variable
+`MISTRAL_LOG_LEVEL` to `DEBUG`, `INFO`, `WARN`, `ERROR` or `FATAL`.

data/README.md

@@ -80,15 +80,17 @@ end
 
 In the [`examples`](https://github.com/wilsonsilva/mistral/tree/main/examples) folder, you will find how to do:
 
-| File Name                                                                                                          | Description                                                    |
-|--------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------|
-| [`chat_no_streaming.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/chat_no_streaming.rb)           | How to use the chat endpoint without streaming                 |
-| [`chat_with_streaming.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/chat_with_streaming.rb)       | How to use the chat endpoint with streaming                    |
-| [`chatbot_with_streaming.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/chatbot_with_streaming.rb) | A simple interactive chatbot using streaming                   |
-| [`embeddings.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/embeddings.rb)                         | How to use the embeddings endpoint                             |
-| [`function_calling.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/function_calling.rb)             | How to call functions using the chat endpoint                  |
-| [`json_format.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/json_format.rb)                       | How to request and parse JSON responses from the chat endpoint |
-| [`list_models.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/list_models.rb)                       | How to list available models                                   |
+| File Name                                                                                                                | Description                                                    |
+|--------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------|
+| [`chat_no_streaming.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/chat_no_streaming.rb)                 | How to use the chat endpoint without streaming                 |
+| [`chat_with_streaming.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/chat_with_streaming.rb)             | How to use the chat endpoint with streaming                    |
+| [`chatbot_with_streaming.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/chatbot_with_streaming.rb)       | A simple interactive chatbot using streaming                   |
+| [`code_completion.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/code_completion.rb)                     | How to perform a code completion                               |
+| [`completion_with_streaming.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/completion_with_streaming.rb) | How to perform a code completion with streaming                |
+| [`embeddings.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/embeddings.rb)                               | How to use the embeddings endpoint                             |
+| [`function_calling.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/function_calling.rb)                   | How to call functions using the chat endpoint                  |
+| [`json_format.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/json_format.rb)                             | How to request and parse JSON responses from the chat endpoint |
+| [`list_models.rb`](https://github.com/wilsonsilva/mistral/blob/main/examples/list_models.rb)                             | How to list available models                                   |
 
 ## 🔨 Development
 

data/examples/chatbot_with_streaming.rb

@@ -10,11 +10,12 @@ require 'optparse'
 require 'mistral'
 
 MODEL_LIST = %w[
-  mistral-tiny
-  mistral-small
-  mistral-medium
+  mistral-tiny-latest
+  mistral-small-latest
+  mistral-medium-latest
+  codestral-latest
 ].freeze
-DEFAULT_MODEL = 'mistral-small'
+DEFAULT_MODEL = 'mistral-small-latest'
 DEFAULT_TEMPERATURE = 0.7
 LOG_FORMAT = '%(asctime)s - %(levelname)s - %(message)s'
 # A hash of all commands and their arguments, used for tab completion.

data/examples/code_completion.rb

@@ -0,0 +1,24 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'dotenv/load'
+require 'mistral'
+
+api_key = ENV.fetch('MISTRAL_API_KEY')
+client = Mistral::Client.new(api_key: api_key)
+
+prompt = 'def fibonacci(n: int):'
+suffix = "n = int(input('Enter a number: '))\nprint(fibonacci(n))"
+
+response = client.completion(
+  model: 'codestral-latest',
+  prompt: prompt,
+  suffix: suffix
+)
+
+print <<~COMPLETION
+  #{prompt}
+  #{response.choices[0].message.content}
+  #{suffix}
+COMPLETION

data/examples/completion_with_streaming.rb

@@ -0,0 +1,24 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'dotenv/load'
+require 'mistral'
+
+api_key = ENV.fetch('MISTRAL_API_KEY')
+client = Mistral::Client.new(api_key: api_key)
+
+prompt = 'def fibonacci(n: int):'
+suffix = "n = int(input('Enter a number: '))\nprint(fibonacci(n))"
+
+print(prompt)
+
+client.completion_stream(
+  model: 'codestral-latest',
+  prompt: prompt,
+  suffix: suffix
+).each do |chunk|
+  print(chunk.choices[0].delta.content) unless chunk.choices[0].delta.content.nil?
+end
+
+print(suffix)

data/examples/function_calling.rb

@@ -73,7 +73,7 @@ tools = [
 ]
 
 api_key = ENV.fetch('MISTRAL_API_KEY')
-model = 'mistral-large-latest'
+model = 'mistral-small-latest'
 
 client = Mistral::Client.new(api_key: api_key)
 
@@ -97,7 +97,9 @@ puts "calling function_name: #{function_name}, with function_params: #{function_
 function_result = names_to_functions[function_name].call(function_params['transaction_id'])
 
 messages << response.choices[0].message
-messages << Mistral::ChatMessage.new(role: 'tool', name: function_name, content: function_result)
+messages << Mistral::ChatMessage.new(
+  role: 'tool', name: function_name, content: function_result, tool_call_id: tool_call.id
+)
 
 response = client.chat(model: model, messages: messages, tools: tools)
 

data/lib/mistral/client.rb

@@ -6,7 +6,7 @@ module Mistral
   # Synchronous wrapper around the async client
   class Client < ClientBase
     def initialize(
-      api_key: ENV['MISTRAL_API_KEY'],
+      api_key: nil,
       endpoint: ENDPOINT,
       max_retries: 5,
       timeout: 120
@@ -158,6 +158,73 @@ module Mistral
       raise Mistral::Error.new(message: 'No response received')
     end
 
+    # A completion endpoint that returns a single response.
+    #
+    # @param model [String] model the name of the model to get completion with, e.g. codestral-latest
+    # @param prompt [String] the prompt to complete
+    # @param suffix [String, nil] the suffix to append to the prompt for fill-in-the-middle completion
+    # @param temperature [Float, nil] temperature the temperature to use for sampling, e.g. 0.5.
+    # @param max_tokens [Integer, nil] the maximum number of tokens to generate, e.g. 100. Defaults to nil.
+    # @param top_p [Float, nil] the cumulative probability of tokens to generate, e.g. 0.9. Defaults to nil.
+    # @param random_seed [Integer, nil] the random seed to use for sampling, e.g. 42. Defaults to nil.
+    # @param stop [Array<String>, nil] a list of tokens to stop generation at, e.g. ['/n/n']
+    # @return [ChatCompletionResponse] a response object containing the generated text.
+    #
+    def completion(
+      model:,
+      prompt:,
+      suffix: nil,
+      temperature: nil,
+      max_tokens: nil,
+      top_p: nil,
+      random_seed: nil,
+      stop: nil
+    )
+      request = make_completion_request(
+        prompt:, model:, suffix:, temperature:, max_tokens:, top_p:, random_seed:, stop:
+      )
+      single_response = request('post', 'v1/fim/completions', json: request, stream: false)
+
+      single_response.each do |response|
+        return ChatCompletionResponse.new(**response)
+      end
+
+      raise Error, 'No response received'
+    end
+
+    # An asynchronous completion endpoint that streams responses.
+    #
+    # @param model [String] model the name of the model to get completions with, e.g. codestral-latest
+    # @param prompt [String] the prompt to complete
+    # @param suffix [String, nil] the suffix to append to the prompt for fill-in-the-middle completion
+    # @param temperature [Float, nil] temperature the temperature to use for sampling, e.g. 0.5.
+    # @param max_tokens [Integer, nil] the maximum number of tokens to generate, e.g. 100. Defaults to nil.
+    # @param top_p [Float, nil] the cumulative probability of tokens to generate, e.g. 0.9. Defaults to nil.
+    # @param random_seed [Integer, nil] the random seed to use for sampling, e.g. 42. Defaults to nil.
+    # @param stop [Array<String>, nil] a list of tokens to stop generation at, e.g. ['/n/n']
+    # @return [Enumerator<ChatCompletionStreamResponse>] a generator that yields response objects containing the
+    #   generated text.
+    #
+    def completion_stream(
+      model:,
+      prompt:,
+      suffix: nil,
+      temperature: nil,
+      max_tokens: nil,
+      top_p: nil,
+      random_seed: nil,
+      stop: nil
+    )
+      request = make_completion_request(
+        prompt:, model:, suffix:, temperature:, max_tokens:, top_p:, random_seed:, stop:, stream: true
+      )
+      response = request('post', 'v1/fim/completions', json: request, stream: true)
+
+      response.lazy.map do |json_streamed_response|
+        ChatCompletionStreamResponse.new(**json_streamed_response)
+      end
+    end
+
     private
 
     def request(method, path, json: nil, stream: false, attempt: 1)

data/lib/mistral/client_base.rb

@@ -5,11 +5,15 @@ module Mistral
     attr_reader :endpoint, :api_key, :max_retries, :timeout
 
     def initialize(endpoint:, api_key: nil, max_retries: 5, timeout: 120)
-      @endpoint = endpoint
-      @api_key = api_key
       @max_retries = max_retries
       @timeout = timeout
 
+      api_key = ENV['MISTRAL_API_KEY'] if api_key.nil?
+
+      raise Error, 'API key not provided. Please set MISTRAL_API_KEY environment variable.' if api_key.nil?
+
+      @api_key = api_key
+      @endpoint = endpoint
       @logger = config_logger
 
       # For azure endpoints, we default to the mistral model
@@ -64,6 +68,57 @@ module Mistral
       parsed_messages
     end
 
+    def make_completion_request(
+      prompt:,
+      model: nil,
+      suffix: nil,
+      temperature: nil,
+      max_tokens: nil,
+      top_p: nil,
+      random_seed: nil,
+      stop: nil,
+      stream: false
+    )
+      request_data = {
+        'prompt' => prompt,
+        'suffix' => suffix,
+        'model' => model,
+        'stream' => stream
+      }
+
+      request_data['stop'] = stop unless stop.nil?
+
+      if model.nil?
+        raise Error.new(message: 'model must be provided') if @default_model.nil?
+
+        request_data['model'] = @default_model
+      else
+        request_data['model'] = model
+      end
+
+      request_data.merge!(
+        build_sampling_params(
+          temperature: temperature,
+          max_tokens: max_tokens,
+          top_p: top_p,
+          random_seed: random_seed
+        )
+      )
+
+      @logger.debug("Completion request: #{request_data}")
+
+      request_data
+    end
+
+    def build_sampling_params(max_tokens: nil, random_seed: nil, temperature: nil, top_p: nil)
+      params = {}
+      params['temperature'] = temperature unless temperature.nil?
+      params['max_tokens'] = max_tokens unless max_tokens.nil?
+      params['top_p'] = top_p unless top_p.nil?
+      params['random_seed'] = random_seed unless random_seed.nil?
+      params
+    end
+
     def make_chat_request(
       messages:,
       model: nil,
@@ -115,7 +170,7 @@ module Mistral
 
     def config_logger
       Logger.new($stdout).tap do |logger|
-        logger.level = ENV.fetch('LOG_LEVEL', 'ERROR')
+        logger.level = ENV.fetch('MISTRAL_LOG_LEVEL', 'ERROR')
 
         logger.formatter = proc do |severity, datetime, progname, msg|
           "#{datetime.strftime("%Y-%m-%d %H:%M:%S")} #{severity} #{progname}: #{msg}\n"

data/lib/mistral/models/chat_completion.rb

@@ -43,6 +43,7 @@ module Mistral
     attribute :content, Types::Strict::Array.of(Types::Strict::String) | Types::Strict::String
     attribute? :name, Types::String.optional
     attribute? :tool_calls, Types::Strict::Array.of(ToolCall).optional
+    attribute? :tool_call_id, Types::String.optional
   end
 
   class DeltaMessage < Dry::Struct

data/lib/mistral/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Mistral
-  VERSION = '0.1.0'
+  VERSION = '0.3.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: mistral
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Wilson Silva
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2024-05-04 00:00:00.000000000 Z
+date: 2024-05-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: dry-struct
@@ -108,7 +108,8 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.23'
-description: You can use the Mistral Ruby client to interact with the Mistral AI API.
+description: A 1:1 Ruby port of the official Mistral Python client, with feature and
+  API parity.
 email:
 - wilson.dsigns@gmail.com
 executables: []
@@ -127,6 +128,8 @@ files:
 - examples/chat_no_streaming.rb
 - examples/chat_with_streaming.rb
 - examples/chatbot_with_streaming.rb
+- examples/code_completion.rb
+- examples/completion_with_streaming.rb
 - examples/embeddings.rb
 - examples/function_calling.rb
 - examples/json_format.rb