tsikol 0.1.0

data/docs/guides/sampling.md

@@ -0,0 +1,804 @@
+# Sampling Guide
+
+Sampling enables MCP servers to request text generation from LLMs through the client. This powerful feature allows your tools to leverage AI capabilities for content generation, analysis, and decision-making.
+
+## Table of Contents
+
+1. [What is Sampling?](#what-is-sampling)
+2. [Enabling Sampling](#enabling-sampling)
+3. [Basic Usage](#basic-usage)
+4. [Sampling Parameters](#sampling-parameters)
+5. [Advanced Patterns](#advanced-patterns)
+6. [Error Handling](#error-handling)
+7. [Testing Sampling](#testing-sampling)
+
+## What is Sampling?
+
+Sampling allows MCP servers to:
+- Request text generation from the client's LLM
+- Create AI-powered tools and features
+- Implement intelligent assistants
+- Generate dynamic content
+- Make context-aware decisions
+
+The client handles the actual LLM interaction, while your server provides the prompts.
+
+## Enabling Sampling
+
+### Server Configuration
+
+Enable sampling in your server:
+
+```ruby
+Tsikol.server "my-server" do
+  # Enable sampling capability
+  sampling true
+  
+  # Register tools that use sampling
+  tool AiWriter
+  tool CodeGenerator
+  tool SmartAnalyzer
+end
+```
+
+### Checking Sampling Availability
+
+Tools should check if sampling is available:
+
+```ruby
+class AiPoweredTool < Tsikol::Tool
+  def execute(input:)
+    unless @server.sampling_enabled?
+      return "Sampling is not enabled. Please enable it in the client."
+    end
+    
+    # Use sampling
+    result = @server.sample_text(
+      messages: build_messages(input),
+      temperature: 0.7
+    )
+    
+    result
+  end
+end
+```
+
+## Basic Usage
+
+### Simple Text Generation
+
+```ruby
+class StoryWriter < Tsikol::Tool
+  description "Generate creative stories"
+  
+  parameter :topic do
+    type :string
+    required
+    description "Story topic or theme"
+  end
+  
+  parameter :style do
+    type :string
+    optional
+    default "fantasy"
+    enum ["fantasy", "sci-fi", "mystery", "romance", "horror"]
+  end
+  
+  parameter :length do
+    type :string
+    optional
+    default "short"
+    enum ["micro", "short", "medium", "long"]
+  end
+  
+  def execute(topic:, style: "fantasy", length: "short")
+    messages = [
+      {
+        role: "system",
+        content: {
+          type: "text",
+          text: "You are a creative story writer specializing in #{style} stories."
+        }
+      },
+      {
+        role: "user",
+        content: {
+          type: "text",
+          text: build_prompt(topic, style, length)
+        }
+      }
+    ]
+    
+    # Request generation from LLM
+    response = @server.sample_text(
+      messages: messages,
+      temperature: 0.8,
+      max_tokens: length_to_tokens(length)
+    )
+    
+    if response[:error]
+      "Error generating story: #{response[:error]}"
+    else
+      response[:text]
+    end
+  end
+  
+  private
+  
+  def build_prompt(topic, style, length)
+    "Write a #{length} #{style} story about: #{topic}"
+  end
+  
+  def length_to_tokens(length)
+    case length
+    when "micro" then 200
+    when "short" then 500
+    when "medium" then 1500
+    when "long" then 3000
+    else 500
+    end
+  end
+end
+```
+
+### Code Generation
+
+```ruby
+class CodeGenerator < Tsikol::Tool
+  description "Generate code with AI assistance"
+  
+  parameter :description do
+    type :string
+    required
+    description "What the code should do"
+  end
+  
+  parameter :language do
+    type :string
+    required
+    description "Programming language"
+    
+    complete do |partial|
+      languages = ["ruby", "python", "javascript", "go", "rust", "java"]
+      languages.select { |l| l.start_with?(partial.downcase) }
+    end
+  end
+  
+  parameter :include_tests do
+    type :boolean
+    optional
+    default false
+    description "Include unit tests"
+  end
+  
+  def execute(description:, language:, include_tests: false)
+    messages = build_code_messages(description, language, include_tests)
+    
+    response = @server.sample_text(
+      messages: messages,
+      temperature: 0.3,  # Lower temperature for code
+      max_tokens: 2000
+    )
+    
+    if response[:error]
+      return "Error generating code: #{response[:error]}"
+    end
+    
+    format_code_response(response[:text], language)
+  end
+  
+  private
+  
+  def build_code_messages(description, language, include_tests)
+    system_prompt = <<~PROMPT
+      You are an expert #{language} programmer. 
+      Generate clean, efficient, well-commented code.
+      Follow #{language} best practices and idioms.
+      #{include_tests ? "Include comprehensive unit tests." : ""}
+    PROMPT
+    
+    [
+      {
+        role: "system",
+        content: { type: "text", text: system_prompt }
+      },
+      {
+        role: "user",
+        content: { 
+          type: "text", 
+          text: "Generate #{language} code to: #{description}"
+        }
+      }
+    ]
+  end
+  
+  def format_code_response(code, language)
+    # Extract code blocks if the LLM included markdown
+    if code.include?("```")
+      code = code.scan(/```#{language}?\n(.*?)```/m).flatten.join("\n\n")
+    end
+    
+    code
+  end
+end
+```
+
+## Sampling Parameters
+
+### Temperature
+
+Controls randomness (0.0 - 1.0):
+
+```ruby
+# Deterministic (good for code, analysis)
+@server.sample_text(
+  messages: messages,
+  temperature: 0.0
+)
+
+# Creative (good for stories, brainstorming)
+@server.sample_text(
+  messages: messages,
+  temperature: 0.9
+)
+```
+
+### Max Tokens
+
+Limit response length:
+
+```ruby
+# Short response
+@server.sample_text(
+  messages: messages,
+  max_tokens: 100
+)
+
+# Detailed response
+@server.sample_text(
+  messages: messages,
+  max_tokens: 2000
+)
+```
+
+### Stop Sequences
+
+Stop generation at specific strings:
+
+```ruby
+# Stop at newline for single-line responses
+@server.sample_text(
+  messages: messages,
+  stop_sequences: ["\n"]
+)
+
+# Stop at specific markers
+@server.sample_text(
+  messages: messages,
+  stop_sequences: ["END", "---", "###"]
+)
+```
+
+### Model Hints
+
+Suggest preferred models:
+
+```ruby
+@server.sample_text(
+  messages: messages,
+  model_hint: "claude-3-opus",  # Hint, not requirement
+  temperature: 0.5
+)
+```
+
+## Advanced Patterns
+
+### Multi-Step Generation
+
+Build complex outputs iteratively:
+
+```ruby
+class ReportGenerator < Tsikol::Tool
+  description "Generate comprehensive reports"
+  
+  parameter :data do
+    type :object
+    required
+    description "Data to analyze"
+  end
+  
+  parameter :sections do
+    type :array
+    optional
+    default ["summary", "analysis", "recommendations"]
+  end
+  
+  def execute(data:, sections: ["summary", "analysis", "recommendations"])
+    report = {}
+    context = [initial_context_message(data)]
+    
+    sections.each do |section|
+      log :info, "Generating section: #{section}"
+      
+      # Add request for specific section
+      context << {
+        role: "user",
+        content: {
+          type: "text",
+          text: "Generate the #{section} section based on the data."
+        }
+      }
+      
+      # Generate section
+      response = @server.sample_text(
+        messages: context,
+        temperature: 0.5,
+        max_tokens: 1000
+      )
+      
+      if response[:error]
+        report[section] = "Error generating #{section}: #{response[:error]}"
+      else
+        report[section] = response[:text]
+        
+        # Add response to context for next section
+        context << {
+          role: "assistant",
+          content: {
+            type: "text",
+            text: response[:text]
+          }
+        }
+      end
+    end
+    
+    format_report(report)
+  end
+  
+  private
+  
+  def initial_context_message(data)
+    {
+      role: "system",
+      content: {
+        type: "text",
+        text: "You are a professional analyst. Analyze this data: #{data.to_json}"
+      }
+    }
+  end
+  
+  def format_report(sections)
+    sections.map { |title, content|
+      "## #{title.capitalize}\n\n#{content}"
+    }.join("\n\n")
+  end
+end
+```
+
+### Structured Output Generation
+
+Generate parseable output:
+
+```ruby
+class DataExtractor < Tsikol::Tool
+  description "Extract structured data from text"
+  
+  parameter :text do
+    type :string
+    required
+    description "Text to analyze"
+  end
+  
+  parameter :schema do
+    type :object
+    required
+    description "Expected data structure"
+  end
+  
+  def execute(text:, schema:)
+    messages = [
+      {
+        role: "system",
+        content: {
+          type: "text",
+          text: <<~PROMPT
+            Extract information from text and return ONLY valid JSON.
+            Match this schema: #{schema.to_json}
+            Do not include any explanation or markdown.
+          PROMPT
+        }
+      },
+      {
+        role: "user",
+        content: {
+          type: "text",
+          text: text
+        }
+      }
+    ]
+    
+    response = @server.sample_text(
+      messages: messages,
+      temperature: 0.0,  # Deterministic for structured output
+      max_tokens: 1000
+    )
+    
+    if response[:error]
+      return { error: response[:error] }
+    end
+    
+    # Parse JSON response
+    begin
+      JSON.parse(response[:text])
+    rescue JSON::ParserError => e
+      {
+        error: "Failed to parse response as JSON",
+        raw_response: response[:text]
+      }
+    end
+  end
+end
+```
+
+### Streaming Generation
+
+For long-running generations:
+
+```ruby
+class StreamingWriter < Tsikol::Tool
+  description "Generate content with progress updates"
+  
+  parameter :prompt do
+    type :string
+    required
+  end
+  
+  parameter :chunks do
+    type :integer
+    optional
+    default 5
+    description "Number of sections to generate"
+  end
+  
+  def execute(prompt:, chunks: 5)
+    output = []
+    
+    chunks.times do |i|
+      chunk_prompt = "#{prompt} (Part #{i + 1}/#{chunks})"
+      
+      response = @server.sample_text(
+        messages: [
+          {
+            role: "user",
+            content: { type: "text", text: chunk_prompt }
+          }
+        ],
+        temperature: 0.7,
+        max_tokens: 500
+      )
+      
+      if response[:error]
+        output << "Error in chunk #{i + 1}: #{response[:error]}"
+      else
+        output << response[:text]
+        
+        # Could notify progress here if supported
+        log :info, "Generated chunk #{i + 1}/#{chunks}"
+      end
+    end
+    
+    output.join("\n\n")
+  end
+end
+```
+
+### Decision Making
+
+Use sampling for intelligent decisions:
+
+```ruby
+class SmartRouter < Tsikol::Tool
+  description "Route requests intelligently"
+  
+  parameter :request do
+    type :string
+    required
+    description "User request"
+  end
+  
+  parameter :available_tools do
+    type :array
+    optional
+    default ["file_manager", "database_query", "web_search"]
+  end
+  
+  def execute(request:, available_tools: ["file_manager", "database_query", "web_search"])
+    messages = [
+      {
+        role: "system",
+        content: {
+          type: "text",
+          text: <<~PROMPT
+            You are a request router. Analyze the user request and determine
+            which tool would best handle it. Available tools:
+            #{format_tool_descriptions(available_tools)}
+            
+            Respond with ONLY the tool name, nothing else.
+          PROMPT
+        }
+      },
+      {
+        role: "user",
+        content: {
+          type: "text",
+          text: request
+        }
+      }
+    ]
+    
+    response = @server.sample_text(
+      messages: messages,
+      temperature: 0.0,
+      max_tokens: 50,
+      stop_sequences: ["\n", " "]
+    )
+    
+    if response[:error]
+      return { error: response[:error] }
+    end
+    
+    selected_tool = response[:text].strip.downcase
+    
+    if available_tools.include?(selected_tool)
+      { tool: selected_tool, confidence: "high" }
+    else
+      { tool: available_tools.first, confidence: "low", raw: selected_tool }
+    end
+  end
+  
+  private
+  
+  def format_tool_descriptions(tools)
+    tool_info = {
+      "file_manager" => "Handles file operations",
+      "database_query" => "Queries databases",
+      "web_search" => "Searches the internet"
+    }
+    
+    tools.map { |tool| "- #{tool}: #{tool_info[tool]}" }.join("\n")
+  end
+end
+```
+
+## Error Handling
+
+### Graceful Degradation
+
+```ruby
+class RobustSampler < Tsikol::Tool
+  def execute(input:)
+    # Try sampling with fallback
+    response = @server.sample_text(
+      messages: build_messages(input),
+      temperature: 0.7,
+      max_tokens: 1000
+    )
+    
+    if response[:error]
+      # Fallback to simpler logic
+      return fallback_handler(input, response[:error])
+    end
+    
+    # Validate response
+    if response[:text].nil? || response[:text].empty?
+      return "Received empty response from AI"
+    end
+    
+    process_response(response[:text])
+  end
+  
+  private
+  
+  def fallback_handler(input, error)
+    log :warning, "Sampling failed, using fallback", error: error
+    
+    # Simple rule-based fallback
+    case input
+    when /analyze/i
+      "Analysis unavailable. Sampling error: #{error}"
+    when /generate/i
+      "Generation unavailable. Sampling error: #{error}"
+    else
+      "AI assistance unavailable. Error: #{error}"
+    end
+  end
+end
+```
+
+### Retry Logic
+
+```ruby
+class RetryingSampler < Tsikol::Tool
+  def execute(input:)
+    max_retries = 3
+    retry_count = 0
+    
+    begin
+      response = @server.sample_text(
+        messages: build_messages(input),
+        temperature: 0.7
+      )
+      
+      if response[:error]
+        raise SamplingError, response[:error]
+      end
+      
+      response[:text]
+      
+    rescue SamplingError => e
+      retry_count += 1
+      
+      if retry_count < max_retries
+        log :warning, "Sampling failed, retrying", 
+            attempt: retry_count, 
+            error: e.message
+        
+        sleep(retry_count) # Exponential backoff
+        retry
+      else
+        "Failed after #{max_retries} attempts: #{e.message}"
+      end
+    end
+  end
+end
+
+class SamplingError < StandardError; end
+```
+
+## Testing Sampling
+
+### Mock Sampling
+
+```ruby
+require 'minitest/autorun'
+require 'tsikol/test_helpers'
+
+class SamplingToolTest < Minitest::Test
+  def setup
+    @server = Tsikol::Server.new(name: "test")
+    @server.sampling true
+    
+    # Mock the sampling method
+    @server.define_singleton_method(:sample_text) do |**params|
+      # Return predictable responses for testing
+      messages = params[:messages]
+      user_content = messages.find { |m| m[:role] == "user" }[:content][:text]
+      
+      case user_content
+      when /error/i
+        { error: "Simulated error" }
+      when /generate.*code/i
+        { text: "def example\n  puts 'Hello'\nend" }
+      else
+        { text: "Test response for: #{user_content}" }
+      end
+    end
+    
+    @server.register_tool_instance(CodeGenerator.new)
+    @client = Tsikol::TestHelpers::TestClient.new(@server)
+  end
+  
+  def test_successful_generation
+    response = @client.call_tool("code_generator", {
+      "description" => "generate hello world",
+      "language" => "ruby"
+    })
+    
+    assert_successful_response(response)
+    result = response.dig(:result, :content, 0, :text)
+    assert_match /def example/, result
+  end
+  
+  def test_error_handling
+    response = @client.call_tool("code_generator", {
+      "description" => "trigger error",
+      "language" => "ruby"
+    })
+    
+    assert_successful_response(response)
+    result = response.dig(:result, :content, 0, :text)
+    assert_match /Error generating code/, result
+  end
+end
+```
+
+### Testing Sampling Parameters
+
+```ruby
+def test_sampling_parameters
+  # Capture sampling calls
+  sampling_calls = []
+  
+  @server.define_singleton_method(:sample_text) do |**params|
+    sampling_calls << params
+    { text: "Generated text" }
+  end
+  
+  @client.call_tool("story_writer", {
+    "topic" => "space adventure",
+    "style" => "sci-fi",
+    "length" => "long"
+  })
+  
+  # Verify parameters
+  assert_equal 1, sampling_calls.length
+  
+  params = sampling_calls.first
+  assert_equal 0.8, params[:temperature]
+  assert_equal 3000, params[:max_tokens]
+  
+  # Check message format
+  messages = params[:messages]
+  assert_equal 2, messages.length
+  assert_equal "system", messages[0][:role]
+  assert_match /sci-fi/, messages[0][:content][:text]
+end
+```
+
+## Best Practices
+
+1. **Clear Prompts**: Write specific, clear prompts for better results
+2. **Temperature Control**: Use appropriate temperature for the task
+3. **Error Handling**: Always handle sampling failures gracefully
+4. **Token Limits**: Be mindful of token limits and costs
+5. **Response Validation**: Validate and sanitize AI responses
+6. **User Feedback**: Show progress for long operations
+7. **Fallback Logic**: Provide alternatives when sampling fails
+
+## Security Considerations
+
+### Input Sanitization
+
+```ruby
+def execute(user_input:)
+  # Sanitize user input before sending to LLM
+  sanitized = user_input.gsub(/[<>]/, '') # Remove potential HTML
+                       .strip
+                       .slice(0, 1000) # Limit length
+  
+  messages = [
+    {
+      role: "user",
+      content: { type: "text", text: sanitized }
+    }
+  ]
+  
+  @server.sample_text(messages: messages)
+end
+```
+
+### Output Validation
+
+```ruby
+def process_ai_response(response)
+  # Validate response format
+  return "Invalid response" unless response.is_a?(String)
+  
+  # Remove potential code injection
+  cleaned = response.gsub(/`.*?`/, '[code removed]')
+  
+  # Check for sensitive data patterns
+  if contains_sensitive_data?(cleaned)
+    return "Response contained sensitive information"
+  end
+  
+  cleaned
+end
+```
+
+## Next Steps
+
+- Add [Middleware](middleware.md) for cross-cutting concerns
+- Implement [Error Handling](../cookbook/error-handling.md)
+- Learn about [Testing](testing.md)
+- Explore [Advanced Patterns](../cookbook/advanced-patterns.md)