aia 0.9.15 → 0.9.17

data/docs/guides/models.md

@@ -373,6 +373,163 @@ premium_models:
 - **Llama 2**: Open-source general purpose
 - **Mixtral**: High-performance open model
 
+## Local Model Providers
+
+### Ollama
+
+[Ollama](https://ollama.ai) enables running open-source AI models locally.
+
+#### Setup
+
+```bash
+# Install Ollama
+brew install ollama  # macOS
+# or download from https://ollama.ai
+
+# Pull models
+ollama pull llama3.2
+ollama pull mistral
+ollama pull qwen2.5-coder
+
+# List available models
+ollama list
+```
+
+#### Usage with AIA
+
+```bash
+# Use Ollama model (prefix with 'ollama/')
+aia --model ollama/llama3.2 my_prompt
+
+# Chat mode
+aia --chat --model ollama/mistral
+
+# List Ollama models from AIA
+aia --model ollama/llama3.2 --chat
+> //models
+
+# Combine with cloud models for comparison
+aia --model ollama/llama3.2,gpt-4o-mini,claude-3-sonnet my_prompt
+```
+
+#### Configuration
+
+```yaml
+# ~/.aia/config.yml
+model: ollama/llama3.2
+
+# Optional: Custom Ollama endpoint
+# Set via environment variable
+export OLLAMA_API_BASE=http://custom-host:11434
+```
+
+#### Popular Ollama Models
+
+- **llama3.2**: Latest Llama model, good general purpose
+- **llama3.2:70b**: Larger version, better quality
+- **mistral**: Fast and efficient
+- **mixtral**: High-performance mixture of experts
+- **qwen2.5-coder**: Specialized for code
+- **codellama**: Code-focused model
+
+### LM Studio
+
+[LM Studio](https://lmstudio.ai) provides a GUI for running local models with OpenAI-compatible API.
+
+#### Setup
+
+1. Download LM Studio from https://lmstudio.ai
+2. Install and launch the application
+3. Browse and download models within LM Studio
+4. Start the local server:
+   - Click "Local Server" tab
+   - Click "Start Server"
+   - Default endpoint: http://localhost:1234/v1
+
+#### Usage with AIA
+
+```bash
+# Use LM Studio model (prefix with 'lms/')
+aia --model lms/qwen/qwen3-coder-30b my_prompt
+
+# Chat mode
+aia --chat --model lms/llama-3.2-3b-instruct
+
+# List LM Studio models from AIA
+aia --model lms/any-loaded-model --chat
+> //models
+
+# Model validation
+# AIA validates model names against LM Studio's loaded models
+# If you specify an invalid model, you'll see:
+#   ❌ 'model-name' is not a valid LM Studio model.
+#
+#   Available LM Studio models:
+#     - lms/qwen/qwen3-coder-30b
+#     - lms/llama-3.2-3b-instruct
+```
+
+#### Configuration
+
+```yaml
+# ~/.aia/config.yml
+model: lms/qwen/qwen3-coder-30b
+
+# Optional: Custom LM Studio endpoint
+# Set via environment variable
+export LMS_API_BASE=http://localhost:1234/v1
+```
+
+#### Tips for LM Studio
+
+- Use the model name exactly as shown in LM Studio
+- Prefix all model names with `lms/`
+- Ensure the local server is running before use
+- LM Studio supports one model at a time (unlike Ollama)
+
+### Comparison: Ollama vs LM Studio
+
+| Feature | Ollama | LM Studio |
+|---------|--------|-----------|
+| **Interface** | Command-line | GUI + CLI |
+| **Model Management** | Via CLI (`ollama pull`) | GUI download |
+| **API Compatibility** | Custom + OpenAI-like | OpenAI-compatible |
+| **Multiple Models** | Yes (switch quickly) | One at a time |
+| **Platform** | macOS, Linux, Windows | macOS, Windows |
+| **Model Format** | GGUF, custom | GGUF |
+| **Best For** | CLI users, automation | GUI users, experimentation |
+
+### Local + Cloud Model Workflows
+
+#### Privacy-First Workflow
+```bash
+# Use local model for sensitive data
+aia --model ollama/llama3.2 --out_file draft.md process_private_data.txt
+
+# Use cloud model for final polish (on sanitized data)
+aia --model gpt-4 --include draft.md refine_output
+```
+
+#### Cost-Optimization Workflow
+```bash
+# Bulk processing with local model (free)
+for file in *.txt; do
+  aia --model ollama/mistral --out_file "${file%.txt}_summary.md" summarize "$file"
+done
+
+# Final review with premium cloud model
+aia --model gpt-4 --include *_summary.md final_report
+```
+
+#### Consensus with Mixed Models
+```bash
+# Get consensus from local and cloud models
+aia --model ollama/llama3.2,ollama/mistral,gpt-4o-mini --consensus decision_prompt
+
+# Or individual responses to compare
+aia --model ollama/llama3.2,lms/qwen-coder,claude-3-sonnet --no-consensus code_review.py
+```
+
 ## Troubleshooting Models
 
 ### Common Issues

data/lib/aia/chat_processor_service.rb

@@ -28,25 +28,37 @@ module AIA
         result = send_to_client(prompt)
       end
 
+      # Debug output to understand what we're receiving
+      puts "[DEBUG ChatProcessor] Result class: #{result.class}" if AIA.config.debug
+      puts "[DEBUG ChatProcessor] Result inspect: #{result.inspect[0..500]}..." if AIA.config.debug
+
       # Preserve token information if available for metrics
       if result.is_a?(String)
+        puts "[DEBUG ChatProcessor] Processing as String" if AIA.config.debug
         { content: result, metrics: nil }
       elsif result.respond_to?(:multi_model?) && result.multi_model?
+        puts "[DEBUG ChatProcessor] Processing as multi-model response" if AIA.config.debug
         # Handle multi-model response with metrics
         {
           content: result.content,
           metrics: nil,  # Individual model metrics handled separately
           multi_metrics: result.metrics_list
         }
-      else
+      elsif result.respond_to?(:content)
+        puts "[DEBUG ChatProcessor] Processing as standard response with content method" if AIA.config.debug
+        # Standard response object with content method
         {
           content: result.content,
           metrics: {
-            input_tokens: result.input_tokens,
-            output_tokens: result.output_tokens,
-            model_id: result.model_id
+            input_tokens: result.respond_to?(:input_tokens) ? result.input_tokens : nil,
+            output_tokens: result.respond_to?(:output_tokens) ? result.output_tokens : nil,
+            model_id: result.respond_to?(:model_id) ? result.model_id : nil
           }
         }
+      else
+        puts "[DEBUG ChatProcessor] Processing as fallback (unexpected type)" if AIA.config.debug
+        # Fallback for unexpected response types
+        { content: result.to_s, metrics: nil }
       end
     end
 
@@ -56,7 +68,10 @@ module AIA
     def send_to_client(conversation)
       maybe_change_model
 
-      AIA.client.chat(conversation)
+      puts "[DEBUG ChatProcessor] Sending conversation to client: #{conversation.inspect[0..500]}..." if AIA.config.debug
+      result = AIA.client.chat(conversation)
+      puts "[DEBUG ChatProcessor] Client returned: #{result.class} - #{result.inspect[0..500]}..." if AIA.config.debug
+      result
     end
 
 

data/lib/aia/directives/models.rb

@@ -30,9 +30,141 @@ module AIA
       end
 
       def self.available_models(args = nil, context_manager = nil)
+          # Check if we're using a local provider
+          current_models = AIA.config.model
+          current_models = [current_models] if current_models.is_a?(String)
+
+          using_local_provider = current_models.any? { |m| m.start_with?('ollama/', 'lms/') }
+
+          if using_local_provider
+            show_local_models(current_models, args)
+          else
+            show_rubyllm_models(args)
+          end
+
+          ""
+        end
+
+      def self.show_local_models(current_models, args)
+          require 'net/http'
+          require 'json'
+
+          puts "\nLocal LLM Models:"
+          puts
+
+          current_models.each do |model_spec|
+            if model_spec.start_with?('ollama/')
+              # Ollama uses its native API, not /v1
+              api_base = ENV.fetch('OLLAMA_API_BASE', 'http://localhost:11434')
+              # Remove /v1 suffix if present
+              api_base = api_base.gsub(%r{/v1/?$}, '')
+              show_ollama_models(api_base, args)
+            elsif model_spec.start_with?('lms/')
+              api_base = ENV.fetch('LMS_API_BASE', 'http://localhost:1234')
+              show_lms_models(api_base, args)
+            end
+          end
+        end
+
+      def self.show_ollama_models(api_base, args)
+          begin
+            uri = URI("#{api_base}/api/tags")
+            response = Net::HTTP.get_response(uri)
+
+            unless response.is_a?(Net::HTTPSuccess)
+              puts "❌ Cannot connect to Ollama at #{api_base}"
+              return
+            end
+
+            data = JSON.parse(response.body)
+            models = data['models'] || []
+
+            if models.empty?
+              puts "No Ollama models found"
+              return
+            end
+
+            puts "Ollama Models (#{api_base}):"
+            puts "-" * 60
+
+            counter = 0
+            models.each do |model|
+              name = model['name']
+              size = model['size'] ? format_bytes(model['size']) : 'unknown'
+              modified = model['modified_at'] ? Time.parse(model['modified_at']).strftime('%Y-%m-%d') : 'unknown'
+
+              entry = "- ollama/#{name} (size: #{size}, modified: #{modified})"
+
+              # Apply query filter if provided
+              if args.nil? || args.empty? || args.any? { |q| entry.downcase.include?(q.downcase) }
+                puts entry
+                counter += 1
+              end
+            end
+
+            puts
+            puts "#{counter} Ollama model(s) available"
+            puts
+          rescue StandardError => e
+            puts "❌ Error fetching Ollama models: #{e.message}"
+          end
+        end
+
+      def self.show_lms_models(api_base, args)
+          begin
+            uri = URI("#{api_base.gsub(%r{/v1/?$}, '')}/v1/models")
+            response = Net::HTTP.get_response(uri)
+
+            unless response.is_a?(Net::HTTPSuccess)
+              puts "❌ Cannot connect to LM Studio at #{api_base}"
+              return
+            end
+
+            data = JSON.parse(response.body)
+            models = data['data'] || []
+
+            if models.empty?
+              puts "No LM Studio models found"
+              return
+            end
+
+            puts "LM Studio Models (#{api_base}):"
+            puts "-" * 60
+
+            counter = 0
+            models.each do |model|
+              name = model['id']
+              entry = "- lms/#{name}"
+
+              # Apply query filter if provided
+              if args.nil? || args.empty? || args.any? { |q| entry.downcase.include?(q.downcase) }
+                puts entry
+                counter += 1
+              end
+            end
+
+            puts
+            puts "#{counter} LM Studio model(s) available"
+            puts
+          rescue StandardError => e
+            puts "❌ Error fetching LM Studio models: #{e.message}"
+          end
+        end
+
+      def self.format_bytes(bytes)
+          units = ['B', 'KB', 'MB', 'GB', 'TB']
+          return "0 B" if bytes.zero?
+
+          exp = (Math.log(bytes) / Math.log(1024)).to_i
+          exp = [exp, units.length - 1].min
+
+          "%.1f %s" % [bytes.to_f / (1024 ** exp), units[exp]]
+        end
+
+      def self.show_rubyllm_models(args)
           query = args
 
-          if 1 == query.size
+          if query && 1 == query.size
             query = query.first.split(',')
           end
 
@@ -42,8 +174,8 @@ module AIA
           puts header + ':'
           puts
 
-          q1 = query.select { |q| q.include?('_to_') } # SMELL: ??
-          q2 = query.reject { |q| q.include?('_to_') }
+          q1 = query ? query.select { |q| q.include?('_to_') } : []
+          q2 = query ? query.reject { |q| q.include?('_to_') } : []
 
           counter = 0
 
@@ -75,8 +207,6 @@ module AIA
           puts if counter > 0
           puts "#{counter} LLMs matching your query"
           puts
-
-          ""
         end
 
       def self.help(args = nil, context_manager = nil)