ollama-ruby 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9f88e75feb900230387f4b960f5762daa8622af0f9562c0145d6b22c4a01fb3d
-  data.tar.gz: 4c6fea6ddc54c83ac6d003d4e5f3ccb6b90195b79a97aea6d98a95e32db36f99
+  metadata.gz: 510c31683b2251118a7c3b469620400016b29fd1ea6f17cc86e4f99f62ecea2f
+  data.tar.gz: a00c4275e42002a01a99f874a855386c83e502551360c1f4146eee0b74f2fd08
 SHA512:
-  metadata.gz: 9002b36cd15680f89c50d6f3ee34b3a4e5d77362cd4982d4ca41fc9abbd0bcd1d44483579686f3bf9f20f9e10b1309c28ae8da789ba8ada66c7b5f6384643f87
-  data.tar.gz: e2566f950aea15cf47d60381340db2b0e7a8e3a2fd79aaa8ee421f6cbd367baeae0f64b24e5d9baa801bd715695edf80c5a88d0ba41867c9358bc3076d1ed112
+  metadata.gz: 17e50d40e4c24b56b5c2923f0b73f3e4294587b61a302fbaf4cd8f891e879eb435541e1f7e195a4f88352fd99aa80ff44c8577b8c195d7ef31af1c33229018b7
+  data.tar.gz: a18bcef82e9481b75fee4a4c88598a7a374c48f841e9870b39174c0d1f5f235f1173bdbff939182481d431e4e9c8c85501be4f5ef917ca02865692cfc39fde5a

data/CHANGES.md

@@ -0,0 +1,105 @@
+# Changes
+
+## 2024-09-05 v0.3.0
+
+* **New Features**
+  * Created new file `ollama_cli` with Ollama CLI functionality.
+  * Added executable `ollama_cli` to s.executables in ollama-ruby.gemspec.
+  * Added `find_where` method in `documents.rb` to filter records by text size
+    and count.
+  * Added test for `find_where` method in `documents_spec.rb`.
+  * Features for `ollama_chat`
+      * Added `found_texts_count` option to `OllamaChatConfig`.
+      * Implemented `parse_rss` method for RSS feeds and `parse_atom` method
+        for Atom feeds.
+      * Added links to titles in RSS feed item summaries and Atom feed item
+        summaries.
+      * Updated `parse_source` method to handle different content types,
+        including HTML, XML, and RSS/Atom feeds.
+      * Added `/web [n] query` command to search web and return n or 1 results
+        in chat interface.
+* **Improvements**
+  * Improved validation for system prompts
+  * Extracted file argument handling into a separate module and method
+  * Added default value for config or model system prompt
+  * Improved input validation for `system_prompt` path
+  * Updated collection clearing logic to accept optional tags parameter
+  * Updated `Tags` class to overload `to_a` method for converting to array of
+    strings
+
+## 2024-09-03 v0.2.0
+
+### Changes
+
+* **Added Web Search Functionality to `ollama_chat`**
+	+ Added `/web` command to fetch search results from DuckDuckGo
+	+ Updated `/summarize` command to handle cases where summarization fails
+	+ Fix bug in parsing content type of source document
+* **Refactored Options Class and Usage**
+	+ Renamed `options` variable to use `Options[]` method in ollama_chat script
+	+ Added `[](value)` method to Ollama::Options class for casting hashes
+	+ Updated options_spec.rb with tests for casting hashes and error handling
+* **Refactored Web Search Command**
+	+ Added support for specifying a page number in `/web` command
+	+ Updated regular expression to match new format
+	+ Passed page number as an argument to `search_web` method
+	+ Updated content string to reference the query and sources correctly
+* **DTO Class Changes**
+	+ Renamed `json_create` method to `from_hash` in Ollama::DTO class
+	+ Updated `as_json` method to remove now unnecessary hash creation
+* **Message and Tool Spec Changes**
+	+ Removed `json_class` from JSON serialization in message_spec
+	+ Removed `json_class` from JSON serialization in tool_spec
+* **Command Spec Changes**
+	+ Removed `json_class` from JSON serialization in various command specs (e.g. generate_spec, pull_spec, etc.)
+* **Miscellaneous Changes**
+	+ Improved width calculation for text truncation
+	+ Updated FollowChat class to display evaluation statistics
+	+ Update OllamaChatConfig to use EOT instead of end for heredoc syntax
+	+ Add .keep file to tmp directory
+
+## 2024-08-30 v0.1.0
+
+### Change Log for New Version
+
+#### Significant Changes
+
+* **Document Splitting and Embedding Functionality**: Added `Ollama::Documents` class with methods for adding documents, checking existence, deleting documents, and finding similar documents.
+	+ Introduced two types of caches: `MemoryCache` and `RedisCache`
+	+ Implemented `SemanticSplitter` class to split text into sentences based on semantic similarity
+* **Improved Ollama Chat Client**: Added support for document embeddings and web/file RAG
+	+ Allowed configuration per yaml file
+	+ Parse user input for URLs or files to send images to multimodal models
+* **Redis Docker Service**: Set `REDIS_URL` environment variable to `redis://localhost:9736`
+	+ Added Redis service to `docker-compose.yml`
+* **Status Display and Progress Updates**: Added infobar.label = response.status when available
+	+ Updated infobar with progress message on each call if total and completed are set
+	+ Display error message from response.error if present
+* **Refactored Chat Commands**: Simplified regular expression patterns for `/pop`, `/save`, `/load`, and `/image` commands
+	+ Added whitespace to some command patterns for better readability
+
+#### Other Changes
+
+* Added `Character` and `RecursiveCharacter` splitter classes to split text into chunks based on character separators
+* Added RSpec tests for the Ollama::Documents class(es)
+* Updated dependencies and added new methods for calculating breakpoint thresholds and sentence embeddings
+* Added 'ollama_update' to executables in Rakefile
+* Started using webmock
+* Refactored chooser and add fetcher specs
+* Added tests for Ollama::Utils::Fetcher
+* Update README.md
+
+## 2024-08-16 v0.0.1
+
+* **New Features**
+	+ Added missing options parameter to Embed command
+	+ Documented new `/api/embed` endpoint
+* **Improvements**
+	+ Improved example in README.md
+* **Code Refactoring**
+	+ Renamed `client` to `ollama` in client and command specs
+	+ Updated expectations to use `ollama` instead of `client`
+
+## 2024-08-12 v0.0.0
+
+  * Start

data/README.md

@@ -43,7 +43,6 @@ ollama_chat [OPTIONS]
   -c CHAT        a saved chat conversation to load
   -C COLLECTION  name of the collection used in this conversation
   -D DOCUMENT    load document and add to collection (multiple)
-  -d             use markdown to display the chat messages
   -v             use voice output
   -h             this help
 ```
@@ -153,19 +152,20 @@ subject - the young, blue-eyed cat.
 The following commands can be given inside the chat, if prefixed by a `/`:
 
 ```
-/paste                             to paste content
-/markdown                          toggle markdown output
-/list                              list the messages of the conversation
-/clear                             clear the conversation messages
-/pop [n]                           pop the last n exchanges, defaults to 1
-/model                             change the model
-/regenerate                        the last answer message
-/collection clear|stats|change|new clear or show stats of current collection
-/summarize source                  summarize the URL/file source's content
-/save filename                     store conversation messages
-/load filename                     load conversation messages
-/quit                              to quit
-/help                              to view this help
+/paste                                   to paste content
+/markdown                                toggle markdown output
+/list                                    list the messages of the conversation
+/clear                                   clear the conversation messages
+/pop [n]                                 pop the last n exchanges, defaults to 1
+/model                                   change the model
+/regenerate                              the last answer message
+/collection clear [tag]|stats|change|new clear or show stats of current collection
+/summarize source                        summarize the URL/file source's content
+/web [n] query                           query web search & return n or 1 results
+/save filename                           store conversation messages
+/load filename                           load conversation messages
+/quit                                    to quit
+/help                                    to view this help
 ```
 
 ### ollama\_console

data/Rakefile

@@ -18,7 +18,8 @@ GemHadar do
      '.utilsrc', '.rspec', *Dir.glob('.github/**/*', File::FNM_DOTMATCH)
   readme      'README.md'
 
-  executables << 'ollama_console' << 'ollama_chat' << 'ollama_update'
+  executables << 'ollama_console' << 'ollama_chat' <<
+    'ollama_update' << 'ollama_cli'
 
   required_ruby_version  '~> 3.1'
 
@@ -36,6 +37,7 @@ GemHadar do
   dependency             'complex_config',        '~> 0.20'
   dependency             'search_ui',             '~> 0.0'
   dependency             'amatch',                '~> 0.4.1'
+  dependency             'pdf-reader',            '~> 2.0'
   development_dependency 'all_images',            '~> 0.4'
   development_dependency 'rspec',                 '~> 3.2'
   development_dependency 'utils'

data/bin/ollama_chat

@@ -4,18 +4,22 @@ require 'ollama'
 include Ollama
 require 'term/ansicolor'
 include Term::ANSIColor
-require 'tins/go'
+require 'tins'
 include Tins::GO
 require 'reline'
 require 'reverse_markdown'
 require 'complex_config'
 require 'fileutils'
+require 'uri'
+require 'nokogiri'
+require 'rss'
+require 'pdf/reader'
 
 class OllamaChatConfig
   include ComplexConfig
   include FileUtils
 
-  DEFAULT_CONFIG = <<~end
+  DEFAULT_CONFIG = <<~EOT
     ---
     url: <%= ENV['OLLAMA_URL'] || 'http://%s' % ENV.fetch('OLLAMA_HOST') %>
     model:
@@ -34,6 +38,7 @@ class OllamaChatConfig
         prompt: 'Represent this sentence for searching relevant passages: %s'
       collection: <%= ENV.fetch('OLLAMA_CHAT_COLLECTION', 'ollama_chat') %>
       found_texts_size: 4096
+      found_texts_count: null
       splitter:
         name: RecursiveCharacter
         chunk_size: 1024
@@ -41,7 +46,7 @@ class OllamaChatConfig
     redis:
       url: <%= ENV.fetch('REDIS_URL', 'null') %>
     debug: <%= ENV['OLLAMA_CHAT_DEBUG'].to_i == 1 ? true : false %>
-  end
+  EOT
 
   def initialize(filename = nil)
     @filename = filename || default_path
@@ -109,14 +114,50 @@ class FollowChat
       end
       @say.call(response)
     end
-    response.done and @output.puts
+    if response.done
+      @output.puts
+      eval_stats = {
+        eval_duration:        Tins::Duration.new(response.eval_duration / 1e9),
+        eval_count:           response.eval_count,
+        prompt_eval_duration: Tins::Duration.new(response.prompt_eval_duration / 1e9),
+        prompt_eval_count:    response.prompt_eval_count,
+        total_duration:       Tins::Duration.new(response.total_duration / 1e9),
+        load_duration:        Tins::Duration.new(response.load_duration / 1e9),
+      }.map { _1 * '=' } * ' '
+      @output.puts '📊 ' + color(111) { Utils::Width.wrap(eval_stats, percentage: 90) }
+    end
     self
   end
 end
 
+def search_web(query, n = 5)
+  query = URI.encode_uri_component(query)
+  url = "https://www.duckduckgo.com/html/?q=#{query}"
+  Ollama::Utils::Fetcher.new.get(url) do |tmp|
+    result = []
+    doc = Nokogiri::HTML(tmp)
+    doc.css('.results_links').each do |link|
+      if n > 0
+        url = link.css('.result__a').first&.[]('href')
+        url.sub!(%r(\A/l/\?uddg=), '')
+        url.sub!(%r(&rut=.*), '')
+        url = URI.decode_uri_component(url)
+        url = URI.parse(url)
+        url.host =~ /duckduckgo\.com/ and next
+        result << url
+        n -= 1
+      else
+        break
+      end
+    end
+    result
+  end
+end
+
 def pull_model_unless_present(model, options, retried = false)
   ollama.show(name: model) { |response|
-    puts "Model #{bold{model}} with architecture #{response.model_info['general.architecture']} found."
+    puts "Model #{bold{model}} with architecture "\
+      "#{response.model_info['general.architecture']} found."
     if system = response.system
       puts "Configured model system prompt is:\n#{italic { system }}"
       return system
@@ -144,7 +185,7 @@ def load_conversation(filename)
     return
   end
   File.open(filename, 'r') do |output|
-    return JSON(output.read, create_additions: true)
+    return JSON(output.read).map { Ollama::Message.from_hash(_1) }
   end
 end
 
@@ -189,19 +230,79 @@ def list_conversation(messages, markdown)
   end
 end
 
+def reverse_markdown(html)
+  ReverseMarkdown.convert(
+    html,
+    unknown_tags: :bypass,
+    github_flavored: true,
+    tag_border: ''
+  )
+end
+
+def parse_rss(source_io)
+  feed = RSS::Parser.parse(source_io, false, false)
+  title = <<~end
+    # #{feed&.channel&.title}
+
+  end
+  feed.items.inject(title) do |text, item|
+    text << <<~end
+      ## [#{item&.title}](#{item&.link})
+
+      updated on #{item&.pubDate}
+
+      #{reverse_markdown(item&.description)}
+
+    end
+  end
+end
+
+def parse_atom(source_io)
+  feed = RSS::Parser.parse(source_io, false, false)
+  title = <<~end
+    # #{feed.title.content}
+
+  end
+  feed.items.inject(title) do |text, item|
+    text << <<~end
+      ## [#{item&.title&.content}](#{item&.link&.href})
+
+      updated on #{item&.updated&.content}
+
+      #{reverse_markdown(item&.content&.content)}
+
+    end
+  end
+end
+
 def parse_source(source_io)
-  case source_io&.content_type&.sub_type
-  when 'html'
-    ReverseMarkdown.convert(
-      source_io.read,
-      unknown_tags: :bypass,
-      github_flavored: true,
-      tag_border: ''
-    )
-  when 'plain', 'csv', 'xml'
+  case source_io&.content_type
+  when 'text/html'
+    reverse_markdown(source_io.read)
+  when 'text/xml'
+    if source_io.readline =~ %r(^\s*<rss\s)
+      source_io.rewind
+      return parse_rss(source_io)
+    end
+    source_io.rewind
+    source_io.read
+  when %r(\Atext/)
+    source_io.read
+  when 'application/rss+xml'
+    parse_rss(source_io)
+  when 'application/atom+xml'
+    parse_atom(source_io)
+  when 'application/json'
     source_io.read
+  when 'application/pdf'
+    reader = PDF::Reader.new(source_io)
+    result = +''
+    reader.pages.each do |page|
+      result << page.text
+    end
+    result
   else
-    STDERR.puts "Cannot import #{source_io.content_type} document."
+    STDERR.puts "Cannot import #{source_io&.content_type} document."
     return
   end
 end
@@ -211,7 +312,7 @@ def import_document(source_io, source)
     STDOUT.puts "Embedding disabled, I won't import any documents, try: /summarize"
     return
   end
-  STDOUT.puts "Importing #{source_io.content_type} document #{source.to_s.inspect}."
+  infobar.puts "Importing #{italic { source_io.content_type }} document #{source.to_s.inspect}."
   text = parse_source(source_io) or return
   text.downcase!
   splitter_config = $config.embedding.splitter
@@ -290,7 +391,7 @@ def parse_content(content, images)
         case source_io&.content_type&.media_type
         when 'image'
           add_image(images, source_io, source)
-        when 'text'
+        when 'text', 'application'
           import_document(source_io, source)
         else
           STDERR.puts(
@@ -354,19 +455,20 @@ end
 
 def display_chat_help
   puts <<~end
-    /paste                             to paste content
-    /markdown                          toggle markdown output
-    /list                              list the messages of the conversation
-    /clear                             clear the conversation messages
-    /pop [n]                           pop the last n exchanges, defaults to 1
-    /model                             change the model
-    /regenerate                        the last answer message
-    /collection clear|stats|change|new clear or show stats of current collection
-    /summarize source                  summarize the URL/file source's content
-    /save filename                     store conversation messages
-    /load filename                     load conversation messages
-    /quit                              to quit
-    /help                              to view this help
+    /paste                                   to paste content
+    /markdown                                toggle markdown output
+    /list                                    list the messages of the conversation
+    /clear                                   clear the conversation messages
+    /pop [n]                                 pop the last n exchanges, defaults to 1
+    /model                                   change the model
+    /regenerate                              the last answer message
+    /collection clear [tag]|stats|change|new clear or show stats of current collection
+    /summarize source                        summarize the URL/file source's content
+    /web [n] query                           query web search & return n or 1 results
+    /save filename                           store conversation messages
+    /load filename                           load conversation messages
+    /quit                                    to quit
+    /help                                    to view this help
   end
 end
 
@@ -381,7 +483,6 @@ def usage
       -c CHAT        a saved chat conversation to load
       -C COLLECTION  name of the collection used in this conversation
       -D DOCUMENT    load document and add to collection (multiple)
-      -d             use markdown to display the chat messages
       -v             use voice output
       -h             this help
 
@@ -393,7 +494,7 @@ def ollama
   $ollama
 end
 
-opts = go 'f:u:m:s:c:C:D:dvh'
+opts = go 'f:u:m:s:c:C:D:vh'
 
 config = OllamaChatConfig.new(opts[?f])
 $config = config.config
@@ -407,13 +508,13 @@ base_url = opts[?u] || $config.url
 $ollama      = Client.new(base_url:, debug: $config.debug)
 
 model        = choose_model(opts[?m], $config.model.name)
-options      = $config.model.options
+options      = Options[$config.model.options]
 model_system = pull_model_unless_present(model, options)
 messages     = []
 
 if $config.embedding.enabled
   embedding_model         = $config.embedding.model.name
-  embedding_model_options = $config.embedding.model.options
+  embedding_model_options = Options[$config.embedding.model.options]
   pull_model_unless_present(embedding_model, embedding_model_options)
   collection = opts[?C] || $config.embedding.collection
   $documents = Documents.new(
@@ -456,23 +557,15 @@ end
 if voice = ($config.voice if opts[?v])
   puts "Using voice #{bold{voice}} to speak."
 end
-
-markdown = set_markdown(opts[?d] || $config.markdown)
+markdown = set_markdown($config.markdown)
 
 if opts[?c]
   messages.concat load_conversation(opts[?c])
 else
-  system = nil
-  if system_prompt_file = opts[?s]
-    system = File.read(system_prompt_file)
-  end
-  system ||= $config.system
-
-  if system
+  if system = Ollama::Utils::FileArgument.
+      get_file_argument(opts[?s], default: $config.system? || model_system)
     messages << Message.new(role: 'system', content: system)
     puts "Configured system prompt is:\n#{italic { system }}"
-  elsif model_system.present?
-    puts "Using model system prompt."
   end
 end
 
@@ -481,9 +574,10 @@ puts "\nType /help to display the chat help."
 images = []
 loop do
   parse_content = true
-
   input_prompt = bold { color(172) { message_type(images) + " user" } } + bold { "> " }
-  case content = Reline.readline(input_prompt, true)&.chomp
+  content = Reline.readline(input_prompt, true)&.chomp
+
+  case content
   when %r(^/paste$)
     puts bold { "Paste your content and then press C-d!" }
     content = STDIN.read
@@ -500,11 +594,18 @@ loop do
     messages.clear
     puts "Cleared messages."
     next
-  when %r(^/collection (clear|stats|change|new)$)
-    case $1
+  when %r(^/collection\s+(clear|stats|change|new)(?:\s+(.+))?$)
+    command, arg = $1, $2
+    case command
     when 'clear'
-      $documents.clear
-      puts "Cleared collection #{bold{collection}}."
+      tags = arg.present? ? arg.sub(/\A#*/, '') : nil
+      if tags
+        $documents.clear(tags:)
+        puts "Cleared tag ##{tags} from collection #{bold{collection}}."
+      else
+        $documents.clear
+        puts "Cleared collection #{bold{collection}}."
+      end
     when 'stats'
       collection_stats
     when 'change'
@@ -518,7 +619,7 @@ loop do
   when %r(^/pop?(?:\s+(\d*))?$)
     n = $1.to_i.clamp(1, Float::INFINITY)
     r =  messages.pop(2 * n)
-    m = r.size
+    m = r.size / 2
     puts "Popped the last #{m} exchanges."
     next
   when %r(^/model$)
@@ -534,7 +635,15 @@ loop do
     end
   when %r(^/summarize\s+(.+))
     parse_content = false
-    content       = summarize($1)
+    content       = summarize($1) or next
+  when %r(^/web\s+(?:(\d+)\s+)?(.+)$)
+    parse_content = true
+    urls = search_web($2, $1.to_i)
+    content = <<~end
+      Answer the the query #{$2.inspect} using these sources:
+
+      #{urls * ?\n}
+    end
   when %r(^/save\s+(.+)$)
     save_conversation($1, messages)
     puts "Saved conversation to #$1."
@@ -557,19 +666,18 @@ loop do
                     [ content, Utils::Tags.new ]
                   end
 
-  if $config.embedding.enabled
-    records = $documents.find(
+  if $config.embedding.enabled && content
+    records = $documents.find_where(
       content.downcase,
       tags:,
-      prompt: $config.embedding.model.prompt?
+      prompt:     $config.embedding.model.prompt?,
+      text_size:  $config.embedding.found_texts_size?,
+      text_count: $config.embedding.found_texts_count?,
     )
-    s, found_texts_size = 0, $config.embedding.found_texts_size
-    records = records.take_while {
-      (s += _1.text.size) <= found_texts_size
-    }
     found_texts = records.map(&:text)
     unless found_texts.empty?
-      content += "\nConsider these chunks for your answer:\n#{found_texts.join("\n\n---\n\n")}"
+      content += "\nConsider these chunks for your answer:\n"\
+        "#{found_texts.join("\n\n---\n\n")}"
     end
   end
 
@@ -577,15 +685,17 @@ loop do
   handler = FollowChat.new(messages:, markdown:, voice:)
   ollama.chat(model:, messages:, options:, stream: true, &handler)
 
-  puts records.map { |record|
-    link = if record.source =~ %r(\Ahttps?://)
-             record.source
-           else
-             'file://%s' % File.expand_path(record.source)
-           end
-    [ link, record.tags.first ]
-  }.uniq.map { |l, t| hyperlink(l, t) }.join(' ')
-  $config.debug and jj messages
+  if records
+    puts records.map { |record|
+      link = if record.source =~ %r(\Ahttps?://)
+               record.source
+             else
+               'file://%s' % File.expand_path(record.source)
+             end
+      [ link, record.tags.first ]
+    }.uniq.map { |l, t| hyperlink(l, t) }.join(' ')
+    $config.debug and jj messages
+  end
 rescue Interrupt
   puts "Type /quit to quit."
 end

data/bin/ollama_cli

@@ -0,0 +1,68 @@
+#!/usr/bin/env ruby
+
+require 'ollama'
+include Ollama
+include Ollama::Utils::FileArgument
+require 'tins'
+include Tins::GO
+require 'json'
+
+def usage
+  puts <<~end
+    #{File.basename($0)} [OPTIONS]
+
+      -u URL         the ollama base url, OLLAMA_URL
+      -m MODEL       the ollama model to chat with, OLLAMA_MODEL
+      -M OPTIONS     the ollama model options to use, OLLAMA_MODEL_OPTIONS
+      -s SYSTEM      the system prompt to use as a file, OLLAMA_SYSTEM
+      -p PROMPT      the user prompt to use as a file, OLLAMA_PROMPT
+      -H HANDLER     the handler to use for the response, defaults to Print
+      -S             use streaming for generation
+      -h             this help
+
+  end
+  exit 0
+end
+
+opts = go 'u:m:M:s:p:H:Sh', defaults: { ?H => 'Print', ?M => '{}' }
+
+opts[?h] and usage
+
+base_url = opts[?u] || ENV['OLLAMA_URL'] || 'http://%s' % ENV.fetch('OLLAMA_HOST')
+model    = opts[?m] || ENV.fetch('OLLAMA_MODEL', 'llama3.1')
+options  = Ollama::Options.from_hash(JSON(
+  get_file_argument(opts[?M], default: ENV['OLLAMA_MODEL_OPTIONS'])
+))
+system   = get_file_argument(opts[?s], default: ENV['OLLAMA_SYSTEM'])
+prompt   = get_file_argument(opts[?p], default: ENV['OLLAMA_PROMPT'])
+
+if prompt.nil?
+  prompt = STDIN.read
+elsif c = prompt.scan('%s').size
+  case c
+  when 0
+  when 1
+    prompt = prompt % STDIN.read
+  else
+    STDERR.puts "Found more than one plaeceholder %s. => Ignoring."
+  end
+end
+
+if ENV['DEBUG'].to_i == 1
+  puts <<~EOT
+    base_url = #{base_url.inspect}
+    model    = #{model.inspect}
+    system   = #{system.inspect}
+    prompt   = #{prompt.inspect}
+    options  = #{options.to_json}
+  EOT
+end
+
+Client.new(base_url:, read_timeout: 120).generate(
+  model:,
+  system:,
+  prompt:,
+  options:,
+  stream: !!opts[?S],
+  &Object.const_get(opts[?H])
+)

data/bin/ollama_console

@@ -5,8 +5,13 @@ include Ollama
 require 'irb'
 require 'irb/history'
 
-base_url = ENV['OLLAMA_URL'] || 'http://%s' % ENV.fetch('OLLAMA_HOST')
-ollama = Client.new(base_url:)
+def base_url
+  ENV['OLLAMA_URL'] || 'http://%s' % ENV.fetch('OLLAMA_HOST')
+end
+
+def ollama
+  $ollama ||= Client.new(base_url:)
+end
 IRB.setup nil
 IRB.conf[:MAIN_CONTEXT] = IRB::Irb.new.context
 IRB.conf[:HISTORY_FILE] = File.join(ENV.fetch('HOME'), '.ollama_console-history')