ollama_chat 0.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 7968cdb60fc0612f59f3c240927a4a11f21ad664009270d614a31a119e818f17
+  data.tar.gz: f27240bd0371ecb4c5e2b20a73f2b94e3091ee3c6e230b313c89b39112be6096
+SHA512:
+  metadata.gz: d665a8f4e8182b0b6a18f3daf86ce7a464632c2dfe3c608332528d10502ac65f4c3663cd7a6ebc18439f4c1156a183ce0962171d3bfe326b074f539fba51cee4
+  data.tar.gz: fa7a6384dabf4204a128ae56ef43c3fe22cbe2842f69ffa5c5d1db16133d64767541be616e6219ac4923cde21d4441b6e8f1a1f24dea726ed27dbbca35043d6a

data/.all_images.yml

@@ -0,0 +1,17 @@
+dockerfile: |-
+  RUN apk add --no-cache build-base git
+  RUN gem update --system
+  RUN gem install gem_hadar bundler
+
+script: &script |-
+  echo -e "\e[1m"
+  ruby -v
+  rm -f Gemfile.lock
+  bundle install --jobs=$(getconf _NPROCESSORS_ONLN) --full-index
+  echo -e "\e[0m"
+  rake test
+
+images:
+  ruby:3.4-alpine: *script
+  ruby:3.3-alpine: *script
+  ruby:3.2-alpine: *script

data/.gitignore

@@ -0,0 +1,9 @@
+.*.sw[pon]
+.AppleDouble
+.bundle
+.yardoc
+Gemfile.lock
+corpus
+coverage
+pkg
+tags

data/Gemfile

@@ -0,0 +1,5 @@
+# vim: set filetype=ruby et sw=2 ts=2:
+
+source 'https://rubygems.org'
+
+gemspec

data/README.md

@@ -0,0 +1,159 @@
+# OllamaChat - Ruby Chat Bot for Ollama
+
+## Description
+
+**ollama_chat** is a chat client, that can be used to connect to an ollama
+server and enter chat conversations with the LLMs provided by it.
+
+## Installation (gem)
+
+To install **ollama_chat**, you can type
+
+```
+gem install ollama_chat
+```
+
+in your terminal.
+
+## Usage
+
+It can be started with the following arguments:
+
+```
+Usage: ollama_chat [OPTIONS]
+
+  -f CONFIG      config file to read
+  -u URL         the ollama base url, OLLAMA_URL
+  -m MODEL       the ollama model to chat with, OLLAMA_CHAT_MODEL
+  -s SYSTEM      the system prompt to use as a file, OLLAMA_CHAT_SYSTEM
+  -c CHAT        a saved chat conversation to load
+  -C COLLECTION  name of the collection used in this conversation
+  -D DOCUMENT    load document and add to embeddings collection (multiple)
+  -M             use (empty) MemoryCache for this chat session
+  -E             disable embeddings for this chat session
+  -V             display the current version number and quit
+  -h             this help
+```
+
+The base URL can be either set by the environment variable `OLLAMA_URL` or it
+is derived from the environment variable `OLLAMA_HOST`. The default model to
+connect can be configured in the environment variable `OLLAMA_MODEL`.
+
+The YAML config file is stored in `$XDG_CONFIG_HOME/ollama_chat/config.yml` and
+you can use it for more complex settings.
+
+### Example: Setting a system prompt
+
+Some settings can be passed as arguments as well, e. g. if you want to choose a
+specific system prompt:
+
+```
+$ ollama_chat -s sherlock.txt
+Model with architecture llama found.
+Connecting to llama3.1@http://ollama.local.net:11434 now…
+Configured system prompt is:
+You are Sherlock Holmes and the user is your new client, Dr. Watson is also in
+the room. You will talk and act in the typical manner of Sherlock Holmes do and
+try to solve the user's case using logic and deduction.
+
+Type /help to display the chat help.
+📨 user:
+Good morning.
+📨 assistant:
+Ah, good morning, my dear fellow! It is a pleasure to make your acquaintance. I
+am Sherlock Holmes, the renowned detective, and this is my trusty sidekick, Dr.
+Watson. Please, have a seat and tell us about the nature of your visit. What
+seems to be the problem that has brought you to our humble abode at 221B Baker
+Street?
+
+(Watson nods in encouragement as he takes notes)
+
+Now, pray tell, what is it that puzzles you, my dear client? A missing item,
+perhaps? Or a mysterious occurrence that requires clarification? The game, as
+they say, is afoot!
+```
+
+### Example: Using a multimodal model
+
+This example shows how an image like this can be sent to the LLM for multimodal
+analysis:
+
+![cat](spec/assets/kitten.jpg)
+
+```
+$ ollama_chat -m llava-llama3
+Model with architecture llama found.
+Connecting to llava-llama3@http://localhost:11434 now…
+Type /help to display the chat help.
+📸 user> What's on this image? ./spec/assets/kitten.jpg
+📨 assistant:
+The image captures a moment of tranquility featuring a young cat. The cat,
+adorned with gray and white fur marked by black stripes on its face and legs,
+is the central figure in this scene. Its eyes, a striking shade of blue, are
+wide open and directed towards the camera, giving an impression of curiosity or
+alertness.
+
+The cat is comfortably nestled on a red blanket, which contrasts vividly with
+its fur. The blanket, soft and inviting, provides a sense of warmth to the
+image. In the background, partially obscured by the cat's head, is another
+blanket of similar red hue. The repetition of the color adds a sense of harmony
+to the composition.
+
+The cat's position on the right side of the photo creates an interesting
+asymmetry with the camera lens, which occupies the left side of the frame. This
+visual balance enhances the overall composition of the image.
+
+There are no discernible texts or other objects in the image. The focus is
+solely on the cat and its immediate surroundings. The image does not provide
+any information about the location or setting beyond what has been described.
+The simplicity of the scene allows the viewer to concentrate on the main
+subject - the young, blue-eyed cat.
+```
+
+### Chat commands
+
+The following commands can be given inside the chat, if prefixed by a `/`:
+
+```
+/copy                           to copy last response to clipboard
+/paste                          to paste content
+/markdown                       toggle markdown output
+/stream                         toggle stream output
+/location                       toggle location submission
+/voice( change)                 toggle voice output or change the voice
+/list [n]                       list the last n / all conversation exchanges
+/clear                          clear the whole conversation
+/clobber                        clear the conversation and collection
+/drop [n]                        drop the last n exchanges, defaults to 1
+/model                          change the model
+/system                         change system prompt (clears conversation)
+/regenerate                     the last answer message
+/collection( clear|change)      change (default) collection or clear
+/info                           show information for current session
+/config                         output current configuration ("/Users/flori/.config/ollama_chat/config.yml")
+/document_policy                pick a scan policy for document references
+/import source                  import the source's content
+/summarize [n] source           summarize the source's content in n words
+/embedding                      toggle embedding paused or not
+/embed source                   embed the source's content
+/web [n] query                  query web search & return n or 1 results
+/links( clear)                  display (or clear) links used in the chat
+/save filename                  store conversation messages
+/load filename                  load conversation messages
+/quit                           to quit
+/help                           to view this help
+```
+
+## Download
+
+The homepage of this app is located at
+
+* https://github.com/flori/ollama\_chat
+
+## Author
+
+<b>OllamaChat</b> was written by [Florian Frank](mailto:flori@ping.de)
+
+## License
+
+This software is licensed under the <i>MIT</i> license.

data/Rakefile

@@ -0,0 +1,58 @@
+# vim: set filetype=ruby et sw=2 ts=2:
+
+require 'gem_hadar'
+
+GemHadar do
+  name        'ollama_chat'
+  module_type :module
+  author      'Florian Frank'
+  email       'flori@ping.de'
+  homepage    "https://github.com/flori/#{name}"
+  summary     'A command-line interface (CLI) for interacting with an Ollama AI model.'
+  description <<~EOT
+    The app provides a command-line interface (CLI) to an Ollama AI model,
+    allowing users to engage in text-based conversations and generate
+    human-like responses. Users can import data from local files or web pages,
+    which are then processed through three different modes: fully importing the
+    content into the conversation context, summarizing the information for
+    concise reference, or storing it in an embedding vector database for later
+    retrieval based on the conversation.
+  EOT
+
+  test_dir    'spec'
+  ignore      '.*.sw[pon]', 'pkg', 'Gemfile.lock', '.AppleDouble', '.bundle',
+    '.yardoc', 'tags', 'corpus', 'coverage'
+
+  readme      'README.md'
+
+  required_ruby_version  '~> 3.1'
+
+  executables << 'ollama_chat'
+
+  dependency             'excon',                 '~> 1.0'
+  dependency             'ollama-ruby',           '~> 0.14'
+  dependency             'documentrix',           '~> 0.0'
+  dependency             'rss',                   '~> 0.3'
+  dependency             'term-ansicolor',        '~> 1.11'
+  dependency             'redis',                 '~> 5.0'
+  dependency             'mime-types',            '~> 3.0'
+  dependency             'reverse_markdown',      '~> 3.0'
+  dependency             'xdg',                   '~> 7.0'
+  dependency             'kramdown-ansi',         '~> 0.0', '>= 0.0.1'
+  dependency             'complex_config',        '~> 0.22', '>= 0.22.2'
+  dependency             'tins',                  '~> 1.34'
+  dependency             'search_ui',             '~> 0.0'
+  dependency             'amatch',                '~> 0.4.1'
+  dependency             'pdf-reader',            '~> 2.0'
+  dependency             'csv',                   '~> 3.0'
+  development_dependency 'all_images',            '~> 0.6'
+  development_dependency 'rspec',                 '~> 3.2'
+  development_dependency 'kramdown',              '~> 2.0'
+  development_dependency 'webmock'
+  development_dependency 'debug'
+  development_dependency 'simplecov'
+
+  licenses << 'MIT'
+  
+  clobber 'coverage'
+end

data/VERSION

@@ -0,0 +1 @@
+0.0.0

data/bin/ollama_chat

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require 'ollama_chat'
+
+exit OllamaChat::Chat.new.start.to_i

data/lib/ollama_chat/chat.rb

@@ -0,0 +1,398 @@
+require 'tins'
+require 'term/ansicolor'
+require 'reline'
+require 'reverse_markdown'
+require 'complex_config'
+require 'fileutils'
+require 'uri'
+require 'nokogiri'
+require 'rss'
+require 'pdf/reader'
+require 'csv'
+require 'xdg'
+
+class OllamaChat::Chat
+  include Tins::GO
+  include Term::ANSIColor
+  include OllamaChat::DocumentCache
+  include OllamaChat::Switches
+  include OllamaChat::ModelHandling
+  include OllamaChat::Parsing
+  include OllamaChat::SourceFetching
+  include OllamaChat::Dialog
+  include OllamaChat::Information
+  include OllamaChat::Clipboard
+  include OllamaChat::MessageType
+
+  def initialize(argv: ARGV.dup)
+    @opts               = go 'f:u:m:s:c:C:D:MEVh', argv
+    @opts[?h] and exit usage
+    @opts[?V] and exit version
+    @ollama_chat_config = OllamaChat::OllamaChatConfig.new(@opts[?f])
+    self.config         = @ollama_chat_config.config
+    setup_switches(config)
+    base_url         = @opts[?u] || config.url
+    @ollama          = Ollama::Client.new(
+      base_url:   base_url,
+      debug:      config.debug,
+      user_agent:
+    )
+    @document_policy = config.document_policy
+    @model           = choose_model(@opts[?m], config.model.name)
+    @model_options   = Ollama::Options[config.model.options]
+    model_system     = pull_model_unless_present(@model, @model_options)
+    @embedding_enabled.set(config.embedding.enabled && !@opts[?E])
+    @messages        = OllamaChat::MessageList.new(self)
+    if @opts[?c]
+      @messages.load_conversation(@opts[?c])
+    else
+      default = config.system_prompts.default? || model_system
+      if @opts[?s] =~ /\A\?/
+        change_system_prompt(default, system: @opts[?s])
+      else
+        system = OllamaChat::Utils::FileArgument.get_file_argument(@opts[?s], default:)
+        system.present? and @messages.set_system_prompt(system)
+      end
+    end
+    @documents     = setup_documents
+    @cache         = setup_cache
+    @current_voice = config.voice.default
+    @images        = []
+  end
+
+  attr_reader :ollama
+
+  attr_reader :documents
+
+  def links
+    @links ||= Set.new
+  end
+
+  class << self
+    attr_accessor :config
+  end
+
+  def config=(config)
+    self.class.config = config
+  end
+
+  def config
+    self.class.config
+  end
+
+  def start
+    info
+    STDOUT.puts "\nType /help to display the chat help."
+
+    loop do
+      parse_content = true
+      input_prompt = bold { color(172) { message_type(@images) + " user" } } + bold { "> " }
+      content = Reline.readline(input_prompt, true)&.chomp
+
+      case content
+      when %r(^/copy$)
+        copy_to_clipboard
+        next
+      when %r(^/paste$)
+        content = paste_from_input
+      when %r(^/markdown$)
+        @markdown.toggle
+        next
+      when %r(^/stream$)
+        @stream.toggle
+        next
+      when %r(^/location$)
+        @location.toggle
+        next
+      when %r(^/voice(?:\s+(change))?$)
+        if $1 == 'change'
+          change_voice
+        else
+          @voice.toggle
+        end
+        next
+      when %r(^/list(?:\s+(\d*))?$)
+        last = 2 * $1.to_i if $1
+        @messages.list_conversation(last)
+        next
+      when %r(^/clear$)
+        @messages.clear
+        STDOUT.puts "Cleared messages."
+        next
+      when %r(^/clobber$)
+        if ask?(prompt: 'Are you sure to clear messages and collection? (y/n) ') =~ /\Ay/i
+          @messages.clear
+          @documents.clear
+          links.clear
+          STDOUT.puts "Cleared messages and collection #{bold{@documents.collection}}."
+        else
+          STDOUT.puts 'Cancelled.'
+        end
+        next
+      when %r(^/drop(?:\s+(\d*))?$)
+        @messages.drop($1)
+        @messages.list_conversation(2)
+        next
+      when %r(^/model$)
+        @model = choose_model('', @model)
+        next
+      when %r(^/system$)
+        change_system_prompt(@system)
+        info
+        next
+      when %r(^/regenerate$)
+        if content = @messages.second_last&.content
+          content.gsub!(/\nConsider these chunks for your answer.*\z/, '')
+          @messages.drop(2)
+        else
+          STDOUT.puts "Not enough messages in this conversation."
+          redo
+        end
+        parse_content = false
+        content
+      when %r(^/collection(?:\s+(clear|change))?$)
+        case $1 || 'change'
+        when 'clear'
+          loop do
+            tags = @documents.tags.add('[EXIT]').add('[ALL]')
+            tag = OllamaChat::Utils::Chooser.choose(tags, prompt: 'Clear? %s')
+            case tag
+            when nil, '[EXIT]'
+              STDOUT.puts "Exiting chooser."
+              break
+            when '[ALL]'
+              if ask?(prompt: 'Are you sure? (y/n) ') =~ /\Ay/i
+                @documents.clear
+                STDOUT.puts "Cleared collection #{bold{@documents.collection}}."
+                break
+              else
+                STDOUT.puts 'Cancelled.'
+                sleep 3
+              end
+            when /./
+              @documents.clear(tags: [ tag ])
+              STDOUT.puts "Cleared tag #{tag} from collection #{bold{@documents.collection}}."
+              sleep 3
+            end
+          end
+        when 'change'
+          choose_collection(@documents.collection)
+        end
+        next
+      when %r(^/info$)
+        info
+        next
+      when %r(^/document_policy$)
+        choose_document_policy
+        next
+      when %r(^/import\s+(.+))
+        parse_content = false
+        content       = import($1) or next
+      when %r(^/summarize\s+(?:(\d+)\s+)?(.+))
+        parse_content = false
+        content       = summarize($2, words: $1) or next
+      when %r(^/embedding$)
+        @embedding_paused.toggle(show: false)
+        @embedding.show
+        next
+      when %r(^/embed\s+(.+))
+        parse_content = false
+        content       = embed($1) or next
+      when %r(^/web\s+(?:(\d+)\s+)?(.+))
+        parse_content   = false
+        urls            = search_web($2, $1.to_i)
+        urls.each do |url|
+          fetch_source(url) { |url_io| embed_source(url_io, url) }
+        end
+        urls_summarized = urls.map { summarize(_1) }
+        query   = $2.inspect
+        results = urls.zip(urls_summarized).
+          map { |u, s| "%s as \n:%s" % [ u, s ] } * "\n\n"
+        content = config.prompts.web % { query:, results: }
+      when %r(^/save\s+(.+)$)
+        @messages.save_conversation($1)
+        STDOUT.puts "Saved conversation to #$1."
+        next
+      when %r(^/links(?:\s+(clear))?$)
+        case $1
+        when 'clear'
+          loop do
+            links_options = links.dup.add('[EXIT]').add('[ALL]')
+            link = OllamaChat::Utils::Chooser.choose(links_options, prompt: 'Clear? %s')
+            case link
+            when nil, '[EXIT]'
+              STDOUT.puts "Exiting chooser."
+              break
+            when '[ALL]'
+              if ask?(prompt: 'Are you sure? (y/n) ') =~ /\Ay/i
+                links.clear
+                STDOUT.puts "Cleared all links in list."
+                break
+              else
+                STDOUT.puts 'Cancelled.'
+                sleep 3
+              end
+            when /./
+              links.delete(link)
+              STDOUT.puts "Cleared link from links in list."
+              sleep 3
+            end
+          end
+        when nil
+          if links.empty?
+            STDOUT.puts "List is empty."
+          else
+            Math.log10(links.size).ceil
+            format  = "% #{}s. %s"
+            connect = -> link { hyperlink(link) { link } }
+            STDOUT.puts links.each_with_index.map { |x, i| format % [ i + 1, connect.(x) ] }
+          end
+        end
+        next
+      when %r(^/load\s+(.+)$)
+        @messages.load_conversation($1)
+        STDOUT.puts "Loaded conversation from #$1."
+        next
+      when %r(^/config$)
+        default_pager = ENV['PAGER'].full?
+        if fallback_pager = `which less`.chomp.full? || `which more`.chomp.full?
+          fallback_pager << ' -r'
+        end
+        my_pager = default_pager || fallback_pager
+        rendered = config.to_s
+        Kramdown::ANSI::Pager.pager(
+          lines: rendered.count(?\n),
+          command: my_pager
+        ) do |output|
+          output.puts rendered
+        end
+        next
+      when %r(^/quit$)
+        STDOUT.puts "Goodbye."
+        return
+      when %r(^/)
+        display_chat_help
+        next
+      when ''
+        STDOUT.puts "Type /quit to quit."
+        next
+      when nil
+        STDOUT.puts "Goodbye."
+        return
+      end
+
+      content, tags = if parse_content
+                        parse_content(content, @images)
+                      else
+                        [ content, Documentrix::Utils::Tags.new ]
+                      end
+
+      if @embedding.on? && content
+        records = @documents.find_where(
+          content.downcase,
+          tags:,
+          prompt:     config.embedding.model.prompt?,
+          text_size:  config.embedding.found_texts_size?,
+          text_count: config.embedding.found_texts_count?,
+        )
+        unless records.empty?
+          content += "\nConsider these chunks for your answer:\n\n"\
+            "#{records.map { [ _1.text, _1.tags_set ] * ?\n }.join("\n\n---\n\n")}"
+        end
+      end
+
+      @messages << Ollama::Message.new(role: 'user', content:, images: @images.dup)
+      @images.clear
+      handler = OllamaChat::FollowChat.new(
+        messages: @messages,
+        markdown: @markdown.on?,
+        voice:    (@current_voice if @voice.on?)
+      )
+      ollama.chat(
+        model: @model,
+        messages: @messages,
+        options: @model_options,
+        stream: @stream.on?,
+        &handler
+      )
+      if @embedding.on? && !records.empty?
+        STDOUT.puts "", records.map { |record|
+          link = if record.source =~ %r(\Ahttps?://)
+                   record.source
+                 else
+                   'file://%s' % File.expand_path(record.source)
+                 end
+          [ link, record.tags.first ]
+        }.uniq.map { |l, t| hyperlink(l, t) }.join(' ')
+        config.debug and jj @messages.to_ary
+      end
+    rescue Interrupt
+      STDOUT.puts "Type /quit to quit."
+    end
+    0
+  end
+
+  private
+
+  def setup_documents
+    if @embedding.on?
+      @embedding_model         = config.embedding.model.name
+      @embedding_model_options = Ollama::Options[config.embedding.model.options]
+      pull_model_unless_present(@embedding_model, @embedding_model_options)
+      collection = @opts[?C] || config.embedding.collection
+      documents = Documentrix::Documents.new(
+        ollama:,
+        model:             @embedding_model,
+        model_options:     config.embedding.model.options,
+        database_filename: config.embedding.database_filename || @ollama_chat_config.database_path,
+        collection:        ,
+        cache:             configure_cache,
+        redis_url:         config.redis.documents.url?,
+        debug:             config.debug
+      )
+
+      document_list = @opts[?D].to_a
+      add_documents_from_argv(documents, document_list)
+      documents
+    else
+      Tins::NULL
+    end
+  end
+
+  def add_documents_from_argv(documents, document_list)
+    if document_list.any?(&:empty?)
+      STDOUT.puts "Clearing collection #{bold{documents.collection}}."
+      documents.clear
+      document_list.reject!(&:empty?)
+    end
+    unless document_list.empty?
+      document_list.map! do |doc|
+        if doc =~ %r(\Ahttps?://)
+          doc
+        else
+          File.expand_path(doc)
+        end
+      end
+      STDOUT.puts "Collection #{bold{documents.collection}}: Adding #{document_list.size} documents…"
+      count = 1
+      document_list.each_slice(25) do |docs|
+        docs.each do |doc|
+          fetch_source(doc) do |doc_io|
+            embed_source(doc_io, doc, count:)
+          end
+          count += 1
+        end
+      end
+    end
+  end
+
+  def setup_cache
+    if url = config.redis.expiring.url?
+      Documentrix::Documents::RedisCache.new(
+        prefix: 'Expiring-',
+        url:,
+        ex:     config.redis.expiring.ex,
+      )
+    end
+  end
+end

data/lib/ollama_chat/clipboard.rb

@@ -0,0 +1,23 @@
+module OllamaChat::Clipboard
+  def copy_to_clipboard
+    if message = @messages.last and message.role == 'assistant'
+      copy = `which #{config.copy}`.chomp
+      if copy.present?
+        IO.popen(copy, 'w') do |clipboard|
+          clipboard.write(message.content)
+        end
+        STDOUT.puts "The last response has been copied to the system clipboard."
+      else
+        STDERR.puts "#{config.copy.inspect} command not found in system's path!"
+      end
+    else
+      STDERR.puts "No response available to copy to the system clipboard."
+    end
+    nil
+  end
+
+  def paste_from_input
+    STDOUT.puts bold { "Paste your content and then press C-d!" }
+    STDIN.read
+  end
+end