llm-lsp 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 200411c08bb9e2d5c1bf5e55ebcaf66359f50ff06729c3b310072d46dde0d79d
+  data.tar.gz: 0e677ff3b6c0c35c6f707eeb2ca95359fe74ee516060a48dd9bbf88218f11687
+SHA512:
+  metadata.gz: 60461145a86057ee35e830c6df1d075d9ee30cbca4bc876d0a64fbef86867b43ab64006f177447b78ec40063d4722248e41a6e0bae22c6b8d6f3a14c5287b3f8
+  data.tar.gz: 96792ec7394ef784cebe710e7841c1d296fe0dd11198574fc81d7898e6f6d06110813570686e1a714c42a7732ad20efa67110264b58254a7c7843ae21745e268

data/README.md

@@ -0,0 +1,189 @@
+# llm-lsp.rb
+
+使用 Ruby 利用 LLM 大模型实现的通用补全 LSP Server，支持 Ollama 等兼容 OpenAI API 的后端。
+
+## 特点
+
+1. 支持大模型的 FIM (Fill-In-the-Middle) 能力进行代码补全
+2. 流式补全，支持请求取消时立即中断连接释放 GPU 资源
+3. 请求防抖，同一文档的连续补全请求只处理最新的
+4. 基于 Async 的异步架构，不阻塞编辑器交互
+
+## 技术栈
+
+1. [async](https://github.com/socketry/async) — 异步事件驱动服务框架
+2. [io-stream](https://github.com/socketry/io-stream) — 配合 async 提供高效的缓冲 IO
+3. [ruby-openai](https://github.com/alexrudall/ruby-openai) — OpenAI 兼容的 LLM API 客户端
+
+## 安装
+
+```bash
+gem install llm-lsp
+```
+
+或在 Gemfile 中添加后 `bundle install`：
+
+```ruby
+gem "llm-lsp"
+```
+
+## 使用
+
+```bash
+llm-lsp [options]
+# 或通过 bundle
+bundle exec llm-lsp [options]
+```
+
+### 命令行参数
+
+| 参数 | 说明 |
+|------|------|
+| `-c`, `--config FILE` | 配置文件路径（默认 `~/.config/llm-lsp/llm-lsp.yml`） |
+| `-m`, `--provider NAME` | 选择 provider（覆盖配置文件设定） |
+| `--verbose LEVEL` | 日志级别 (1=ERROR, 2=WARN, 3=INFO, 4=DEBUG) |
+| `--log FILE` | 日志文件路径（默认 STDERR） |
+| `-v`, `--version` | 显示版本号 |
+| `-h`, `--help` | 显示帮助信息 |
+
+### 配置文件
+
+支持 YAML 配置文件，默认路径 `~/.config/llm-lsp/llm-lsp.yml`，可通过 `-c` 指定其他路径。
+
+```yaml
+provider: ollama
+
+providers:
+  ollama:
+    model: qwen2.5-coder:1.5b
+    api_base: http://localhost:11434/v1
+    context_window: 2048
+    tokens_to_clear:
+      - "<|endoftext|>"
+  openai:
+    model: gpt-4
+    api_base: https://api.openai.com/v1
+    access_token: sk-xxxx
+    context_window: 8192
+```
+
+**优先级**（从低到高）：
+1. 配置文件（`~/.config/llm-lsp/llm-lsp.yml`）
+2. `-m` 命令行参数（仅覆盖 provider 选择）
+3. LSP 客户端 `initializationOptions`（最高优先级，同名 provider 完全覆盖）
+
+### LSP 协议配置
+
+#### initializationOptions
+
+通过 LSP 客户端的 `initialize` 请求传入。完整结构：
+
+```jsonc
+{
+  "provider": "ollama",              // 可选，当前使用的 provider 名称
+  "providers": {                     // 可选，provider 定义（支持多个）
+    "ollama": {
+      "model": "qwen2.5-coder:1.5b", // 必填，模型名称
+      "api_base": "http://localhost:11434/v1", // 必填，API 端点
+      "access_token": "",             // 可选，API 密钥（默认空）
+      "context_window": 2048,         // 可选，上下文窗口大小（默认 2048）
+      "fim": {                        // 可选，FIM 特殊标记（默认 null，使用 prompt+suffix 模式）
+        "prefix": "<fim_prefix>",
+        "suffix": "<fim_suffix>",
+        "middle": "<fim_middle>"
+      },
+      "tokenizer_config": {           // 可选，tokenizer 配置（默认 null，回退字符计数）
+        // 三选一：
+        "path": "/path/to/tokenizer.json",         // 本地文件
+        "repository": "Qwen/Qwen2.5-Coder-1.5B",   // HuggingFace Hub
+        "url": "https://example.com/tokenizer.json" // URL 下载
+      },
+      "tokens_to_clear": ["<|endoftext|>"] // 可选，从补全结果中清除的标记（默认 []）
+    }
+  }
+}
+```
+
+当 `initializationOptions` 未提供 providers/provider 时，会使用配置文件中的默认值。
+
+#### workspace/didChangeConfiguration
+
+运行时动态修改配置，参数结构位于 `params.settings`：
+
+```jsonc
+{
+  "provider": "ollama",     // 可选，切换当前 provider
+  "providers": {            // 可选，更新 provider 配置（结构同 initializationOptions.providers）
+    "ollama": {
+      "model": "qwen2.5-coder:7b",
+      "api_base": "http://localhost:11434/v1"
+      // ... 同上所有字段
+    }
+  }
+}
+```
+
+### 编辑器集成
+
+#### coc.nvim
+
+在 `coc-settings.json` 中添加：
+
+```json
+{
+  "languageserver": {
+    "llm-lsp": {
+      "command": "llm-lsp",
+      "args": ["--log", "/tmp/llm-lsp.log", "--verbose", "4"],
+      "filetypes": ["*"],
+      "initializationOptions": {
+        "provider": "ollama",
+        "providers": {
+          "ollama": {
+            "model": "qwen2.5-coder:1.5b",
+            "api_base": "http://localhost:11434/v1"
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+如果已在配置文件中定义了 providers，`initializationOptions` 可以省略 providers 部分：
+
+```json
+{
+  "languageserver": {
+    "llm-lsp": {
+      "command": "llm-lsp",
+      "args": ["-c", "/path/to/config.yml", "--log", "/tmp/llm-lsp.log"],
+      "filetypes": ["*"]
+    }
+  }
+}
+```
+
+## 测试
+
+项目使用 Minitest 进行集成测试，通过启动 LSP 服务器子进程并进行 JSON-RPC 通信验证。
+
+```bash
+# 运行全部测试
+bundle exec rake test
+bundle exec ruby test/test_lsp.rb
+
+# 运行单个测试
+bundle exec ruby test/test_lsp.rb --name test_inline_completion
+```
+
+**注意：**
+- 部分测试依赖 Ollama 在 `localhost:11434` 运行，不可用时自动 skip
+- Ollama 首次加载模型可能需要 30 秒以上
+- 测试日志输出到 `/tmp/llm-lsp-test.log`
+
+## 依赖
+
+- Ruby (参考 `.ruby-version`)
+- Bundler
+- Ollama 或其他兼容 OpenAI Completions API 的后端

data/bin/llm-lsp

@@ -0,0 +1,109 @@
+#!/usr/bin/env ruby
+
+require "json"
+require "logger"
+require "optparse"
+require "yaml"
+
+require_relative "../lib/llm_lsp"
+
+DEFAULT_CONFIG_PATH = File.join(Dir.home, ".config", "llm-lsp", "llm-lsp.yml")
+
+def parse_options
+  options = {
+    verbose: 1,
+    logfile: nil,
+    config: DEFAULT_CONFIG_PATH,
+    provider: nil,
+    parser: nil,
+  }
+
+  opts = OptionParser.new do |o|
+    o.banner = "Usage: llm-lsp [options]"
+
+    o.on("-c", "--config FILE", String,
+         "Config file path (default: #{DEFAULT_CONFIG_PATH})") do |f|
+      options[:config] = f
+    end
+
+    o.on("-m", "--provider NAME", String,
+         "Select provider by name") do |name|
+      options[:provider] = name
+    end
+
+    o.on("--verbose LEVEL", Integer, "Set verbose level (1-4)") do |v|
+      options[:verbose] = v
+    end
+
+    o.on("--log FILE", String, "Path to log file (default STDERR)") do |f|
+      options[:logfile] = f
+    end
+
+    o.on("-v", "--version", "Show version") do
+      puts "llm-lsp #{LlmLsp::VERSION}"
+      exit 0
+    end
+
+    o.on_tail("-h", "--help", "Show this message") do
+      puts o
+      exit 0
+    end
+  end
+  opts.parse!(ARGV)
+  options[:parser] = opts
+  options
+end
+
+def create_logger(options)
+  logdev = if options[:logfile]
+             file = File.open(options[:logfile], "a")
+             file.sync = true
+             STDERR.reopen(file)
+             file
+           else
+             STDERR
+           end
+
+  logger = Logger.new(logdev)
+  logger.level = case options[:verbose]
+                 when 1 then Logger::ERROR
+                 when 2 then Logger::WARN
+                 when 3 then Logger::INFO
+                 when 4 then Logger::DEBUG
+                 else Logger::INFO
+                 end
+  logger.formatter = proc do |severity, datetime, _prg, msg|
+    if msg.is_a? Exception
+      msg = "#{msg.message} (#{msg.class})\n" << (msg.backtrace || []).join("\n")
+    end
+    timestamp = datetime.strftime("%Y-%m-%dT%H:%M:%S.%L%:z")
+    "#{severity[0]}[#{timestamp}] #{msg}\n"
+  end
+  logger
+rescue => e
+  STDERR.puts "Failed to create logger: #{e.message}"
+  Logger.new(STDERR)
+end
+
+def load_config(path, logger)
+  return {} unless path && File.exist?(path)
+  config = YAML.safe_load_file(path, symbolize_names: true) || {}
+  logger.info("Loaded config from #{path}")
+  config
+rescue => e
+  logger.warn("Failed to load config #{path}: #{e.message}")
+  {}
+end
+
+def main
+  options = parse_options
+  logger = create_logger(options)
+
+  config = load_config(options[:config], logger)
+  config[:provider] = options[:provider].to_sym if options[:provider]
+
+  server = LlmServer.new(logger, config: config)
+  server.run
+end
+
+main

data/lib/core_ext/hash_deep_merge.rb

@@ -0,0 +1,19 @@
+module HashDeepMerge
+  refine Hash do
+    def deep_merge(other_hash)
+      merge(other_hash) do |key, old_val, new_val|
+        if old_val.is_a?(Hash) && new_val.is_a?(Hash)
+          old_val.deep_merge(new_val)
+        elsif old_val.is_a?(Array) && new_val.is_a?(Array)
+          old_val + new_val
+        else
+          new_val
+        end
+      end
+    end
+
+    def deep_merge!(other_hash, &block)
+      replace(deep_merge(other_hash, &block))
+    end
+  end
+end

data/lib/llm_lsp/version.rb

@@ -0,0 +1,3 @@
+module LlmLsp
+  VERSION = "0.2.0"
+end

data/lib/llm_lsp.rb

@@ -0,0 +1,2 @@
+require_relative "llm_lsp/version"
+require_relative "llm_server"

data/lib/llm_server.rb

@@ -0,0 +1,537 @@
+require "openai"
+require_relative "lsp/session"
+require_relative "lsp/json_rpc"
+require_relative "lsp/stdio"
+require_relative "lsp/code"
+
+class Document
+  attr_reader :uri, :text, :position_encoding
+
+  # text 来自 JSON-RPC（JSON 规范 UTF-8），到 Ruby 端已是 UTF-8 字符串
+  # position_encoding 只影响 Position.character 数字的计量单位
+  def initialize(uri, text, position_encoding = "utf-16")
+    @uri = uri
+    @text = text
+    @position_encoding = position_encoding
+  end
+
+  # 将 LSP Position (line, character) 转换为 @text 中的字符偏移量
+  def position_to_offset(lineno, char)
+    offset = 0
+    @text.each_line.with_index do |line, index|
+      return offset + lsp_char_to_chars(line, char) if lineno == index
+      offset += line.length
+    end
+  end
+
+  def slice(range)
+    @text[range]
+  end
+
+  private
+
+  # 将 LSP character offset 转换为行内 Ruby 字符数
+  # utf-16: UTF-16 code unit 数，BMP 字符占 1，补充平面占 2
+  # utf-32: code point 数，等于 Ruby 字符数
+  # utf-8:  字节数，逐字符累加 bytesize 直到达到目标
+  def lsp_char_to_chars(line_text, lsp_char)
+    return lsp_char if @position_encoding == "utf-32"
+
+    consumed = 0
+    line_text.each_char.with_index do |ch, i|
+      return i if consumed >= lsp_char
+      consumed += char_units(ch)
+    end
+    line_text.length
+  end
+
+  def char_units(ch)
+    case @position_encoding
+    when "utf-8"
+      ch.bytesize
+    when "utf-16"
+      ch.ord > 0xFFFF ? 2 : 1
+    else
+      1
+    end
+  end
+end
+
+class LlmServer
+  attr_reader :session, :logger
+
+  def initialize(logger, config: {})
+    @logger = logger
+    @config = config
+    channel = JsonRpc.new(Stdio.new)
+    @session = LspSession.new("LlmLsp", channel, logger)
+    @providers = {}
+    @provider = nil
+
+    @position_encoding = "utf-16"
+    @documents = {}
+    @debounce_delay = 0.2
+    @pending_completions = {}
+
+    # 补全采纳追踪：单协程 + 有序队列
+    # Ruby Hash 保持插入顺序，补全按时间先后发生，expire_at 单调递增，
+    # 因此 Hash 的迭代顺序天然按过期时间从早到晚排列
+    # key: completion_id, value: expire_at (Float, monotonic clock)
+    @pending_accepts = {}
+    @accept_timeout = 30
+    @tokenizer_cache = {}
+
+    setup_handlers
+  end
+
+  def run
+    logger.info("Starting LLM LSP Server...")
+    Sync do |task|
+      task.async { accept_timer_loop }
+      task.async { @session.start }
+    end
+  end
+
+  def setup_handlers
+    session.on_method("initialize") do |msg|
+      handle_initialize(msg)
+    end
+    
+    session.on_method("initialized") do |msg|
+      logger.info("Client initialized.")
+    end
+    
+    session.on_method("textDocument/didOpen") do |msg|
+      params = msg.fetch(:params, {})
+      uri = params.dig(:textDocument, :uri)
+      text = params.dig(:textDocument, :text)
+      @documents[uri] = Document.new(uri, text, @position_encoding)
+      logger.debug("Opened document: #{uri}")
+    end
+
+    session.on_method("textDocument/didChange") do |msg|
+      params = msg.fetch(:params, {})
+      uri = params.dig(:textDocument, :uri)
+      changes = params.dig(:contentChanges)
+      @documents[uri] = Document.new(uri, changes.dig(0, :text), @position_encoding)
+      logger.debug("Updated document: #{uri}")
+    end
+
+    session.on_method("textDocument/didClose") do |msg|
+      uri = msg.dig(:params, :textDocument, :uri)
+      @documents.delete(uri)
+      @pending_completions.delete(uri)&.stop
+      logger.debug("Closed document: #{uri}")
+    end
+
+    session.on_method("textDocument/inlineCompletion") do |msg|
+      handle_inline_completion(msg)
+    end
+
+    session.on_method("workspace/didChangeConfiguration") do |msg|
+      handle_did_change_configuration(msg)
+    end
+
+    session.on_method("workspace/executeCommand") do |msg|
+      handle_execute_command(msg)
+    end
+  end
+
+  def handle_initialize(msg)
+    params = msg.fetch(:params, {})
+    opts = params.fetch(:initializationOptions, {})
+    logger.debug("initializationOptions: #{opts}")
+
+    errors = []
+
+    # 1) 配置文件 providers（低优先级）
+    @config.fetch(:providers, {}).each do |name, conf|
+      if (err = add_provider(name, conf))
+        logger.warn(err)
+        errors << err
+      end
+    end
+
+    # 2) initializationOptions providers（高优先级，同名覆盖）
+    opts.fetch(:providers, {}).each do |name, conf|
+      if (err = add_provider(name, conf))
+        logger.warn(err)
+        errors << err
+      end
+    end
+
+    # provider 选择：initializationOptions > 配置文件（含 -m 覆盖）
+    @provider = if opts.dig(:provider)
+                  opts[:provider].to_sym
+                elsif @config.dig(:provider)
+                  @config[:provider].to_sym
+                else
+                  nil
+                end
+
+    if !@providers.key?(@provider)
+      session.reply(
+        msg[:id],
+        code: Code::INVALID_PARAMS,
+        message: "Provider '#{@provider}' not found, available: #{@providers.keys.join(", ")}"
+      )
+      # 通知客户端初始化过程中有一些非致命错误
+      errors.each { |err| notify_editor(err) }
+      return
+    end
+
+    logger.info("Providers: #{@providers.keys.join(", ")}, active: #{@provider}")
+
+    # position encoding 协商：客户端在 capabilities.general.positionEncodings 声明支持列表
+    # 服务器从中选一个，未声明则默认 utf-16
+    client_encodings = params.dig(:capabilities, :general, :positionEncodings) || []
+    # 优先选 utf-32（最简单），其次 utf-16（最通用）
+    @position_encoding = if client_encodings.include?("utf-32")
+                           "utf-32"
+                         elsif client_encodings.include?("utf-16")
+                           "utf-16"
+                         elsif client_encodings.include?("utf-8")
+                           "utf-8"
+                         else
+                           "utf-16"
+                         end
+    logger.info("Position encoding: #{@position_encoding}")
+
+    session.reply(msg[:id], result: {
+      capabilities: {
+        positionEncoding: @position_encoding,
+        # 0: None, 1: Full, 2: Incremental
+        textDocumentSync: 1, # Full sync TODO: Use incremental sync
+        inlineCompletionProvider: true,
+        executeCommandProvider: {
+          commands: [
+            # 自定义命令，接受 inline 补全
+            "inlineCompletion/accept"
+          ],
+        }
+        #completionProvider: {
+        #  resolveProvider: false,
+        #  #triggerCharacters: ["."]
+        #}
+      },
+      serverInfo: {
+        name: "llm-lsp",
+        version: LlmLsp::VERSION,
+      }
+    })
+
+    # 通知客户端初始化过程中有一些非致命错误
+    errors.each { |err| notify_editor(err) }
+  end
+
+  def handle_did_change_configuration(msg)
+    settings = msg.dig(:params, :settings) || {}
+
+    settings.fetch(:providers, {}).each do |name, provider_conf|
+      if (err = add_provider(name, provider_conf))
+        logger.warn(err)
+        notify_editor(err)
+        next
+      end
+      logger.info("Provider #{name} updated")
+    end
+
+    provider = settings.dig(:provider)&.to_sym
+    return if @provider == provider
+    if !@providers.key?(provider)
+      err = "Unknown provider: '#{provider}', available: #{@providers.keys.join(", ")}"
+      logger.warn(err)
+      notify_editor(err)
+      return
+    end
+
+    @provider = provider
+    logger.info("Switched to provider: #{@provider}")
+    notify_editor("Switched to provider: #{@provider}", type: 3)
+  end
+
+  def current_provider
+    @providers[@provider]
+  end
+
+  def current_client
+    current_provider[:client]
+  end
+
+  # window/logMessage 记录日志
+  # window/showMessage 提示用户
+  # window/showMessageRequest 需要用户交互，
+  #   多了 actions: [ { title: "ok" }, { title: "no" } ]
+  # type: 1 Error, 2 Warn, 3: Info, 4: Log
+  def notify_editor(message, type: 2)
+    session.notify("window/showMessage", params: {
+      type: type,
+      message: "LlmLsp: " + message,
+    })
+  end
+
+  def handle_execute_command(msg)
+    id = msg[:id]
+    params = msg.fetch(:params, {})
+    command = params.dig(:command)
+    arguments = params.dig(:arguments)
+    kind = arguments&.at(0)
+    item = arguments&.at(1)
+
+    unless command == "inlineCompletion/accept" && kind == "llm-lsp" && item
+      return session.reply(id, code: Code::INVALID_PARAMS, message: "Unknown command: #{command}")
+    end
+
+    session.reply(id, result: nil)
+
+    completion_id = item.dig(:id)
+    @pending_accepts.delete(completion_id)
+    logger.info("Completion accepted: #{completion_id}, model: #{item.dig(:model)}")
+  end
+
+  # 记录一个待确认的补全，超时后由 accept_timer_loop 清理并记录拒绝
+  def track_completion(completion_id)
+    @pending_accepts[completion_id] = Process.clock_gettime(Process::CLOCK_MONOTONIC) + @accept_timeout
+  end
+
+  # 常驻 fiber：每秒扫描一次，清理已过期的补全
+  # Ruby Hash 保持插入顺序，expire_at 单调递增，
+  # 从头遍历到第一个未过期的即可 break
+  def accept_timer_loop
+    loop do
+      sleep(1)
+      now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      @pending_accepts.each do |cid, expire_at|
+        break if expire_at > now
+        @pending_accepts.delete(cid)
+        logger.info("Completion rejected (timeout): #{cid}")
+      end
+    end
+  end
+
+  # 注册 provider，返回 nil 成功，String 错误信息失败
+  def add_provider(name, conf)
+    return "Provider '#{name}': config must be a Hash, got #{conf.class}" unless conf.is_a?(Hash)
+    return "Provider '#{name}': missing required field 'model'" unless conf.dig(:model)
+    return "Provider '#{name}': missing required field 'api_base'" unless conf.dig(:api_base)
+
+    client = build_client(conf)
+    return "Provider '#{name}': failed to create client" unless client
+
+    @providers[name] = {
+      name: name,
+      model: conf[:model],
+      api_base: conf[:api_base],
+      access_token: conf.fetch(:access_token, ""),
+      client: client,
+      context_window: conf.fetch(:context_window, 2048),
+      fim: conf.dig(:fim),
+      tokenizer_config: conf.dig(:tokenizer_config),
+      tokens_to_clear: conf.fetch(:tokens_to_clear, []),
+    }
+    nil
+  end
+
+  def build_client(conf)
+    OpenAI::Client.new(
+      access_token: conf.fetch(:access_token, ""),
+      uri_base: conf[:api_base],
+      log_errors: true,
+    )
+  rescue => e
+    logger.error("Failed to create OpenAI client: #{e.message}")
+    nil
+  end
+
+  def load_tokenizer(config)
+    return nil unless config
+    key = config[:path] || config[:repository] || config[:url]
+    return nil unless key
+    return @tokenizer_cache[key] if @tokenizer_cache.key?(key)
+
+    require "tokenizers" unless defined?(Tokenizers)
+
+    tokenizer = if config[:path]
+                  Tokenizers.from_file(config[:path])
+                elsif config[:repository]
+                  Tokenizers.from_pretrained(config[:repository])
+                elsif config[:url]
+                  download_and_load_tokenizer(config[:url])
+                end
+    @tokenizer_cache[key] = tokenizer if tokenizer
+    tokenizer
+  rescue LoadError => e
+    logger.warn("Tokenizers gem not available, falling back to character counting: #{e.message}")
+    @tokenizer_cache[key] = nil
+    nil
+  rescue => e
+    logger.warn("Failed to load tokenizer: #{e.message}")
+    nil
+  end
+
+  def download_and_load_tokenizer(url)
+    require "open-uri"
+    cache_dir = File.join(Dir.home, ".cache", "llm-lsp", "tokenizers")
+    FileUtils.mkdir_p(cache_dir)
+    filename = File.join(cache_dir, Digest::SHA256.hexdigest(url) + ".json")
+    unless File.exist?(filename)
+      URI.open(url) do |remote|
+        File.write(filename, remote.read)
+      end
+    end
+    Tokenizers.from_file(filename)
+  end
+
+  def count_tokens(tokenizer, text)
+    if tokenizer
+      tokenizer.encode(text, add_special_tokens: false).ids.size
+    else
+      text.length
+    end
+  end
+
+  # 参考 llm-ls 的 build_prompt：逐行交替收集 before/after，按 token 计数截断
+  def build_prompt(doc, line, char, tokenizer, context_window, fim)
+    curr = doc.position_to_offset(line, char)
+    before_text = doc.slice(0...curr) || ""
+    after_text = doc.slice(curr..) || ""
+
+    before_lines = before_text.lines
+    after_lines = after_text.lines
+    before_lines = [""] if before_lines.empty?
+    after_lines = [""] if after_lines.empty?
+
+    fim_overhead = fim ? count_tokens(tokenizer, "#{fim[:prefix]}#{fim[:suffix]}#{fim[:middle]}") : 0
+    remaining = context_window - fim_overhead
+
+    collected_before = []
+    collected_after = []
+
+    bi = before_lines.size - 1
+    ai = 0
+
+    while (bi >= 0 || ai < after_lines.size) && remaining > 0
+      if bi >= 0
+        tokens = count_tokens(tokenizer, before_lines[bi])
+        break if tokens > remaining
+        remaining -= tokens
+        collected_before.unshift(before_lines[bi])
+        bi -= 1
+      end
+      if ai < after_lines.size && remaining > 0
+        tokens = count_tokens(tokenizer, after_lines[ai])
+        break if tokens > remaining
+        remaining -= tokens
+        collected_after << after_lines[ai]
+        ai += 1
+      end
+    end
+
+    prefix = collected_before.join
+    suffix = collected_after.join
+
+    if fim
+      prompt = "#{fim[:prefix]}#{prefix}#{fim[:suffix]}#{suffix}#{fim[:middle]}"
+      { prompt: prompt, suffix: nil }
+    else
+      { prompt: prefix, suffix: suffix }
+    end
+  end
+
+  # 防抖 + 流式补全：
+  # - llm 采用流式接口时，取消请求发出后立即关闭链接，释放GPU资源；
+  #   对于api接口非流失只要提交请求就会按照完整生成来计费，无论是否读取了输出，而流式通常会终止推理
+  # - 同一文档的并发请求通过 @pending_completions 做合并，只处理最新请求
+  # - ruby-openai 没有直接的取消方法，但 Async 环境下 task.stop 抛出
+  #   Async::Stop 异常会中断底层 IO 操作，从而中断 HTTP 连接释放 GPU 资源
+  # - 代码需要考虑 Async::Stop 异常，保证异常安全性，
+  #   如果代码不用管理异常也是安全的，那么可以交由框架处理
+  def handle_inline_completion(msg)
+    id = msg[:id]
+    params = msg.fetch(:params, {})
+    uri = params.dig(:textDocument, :uri)
+    position = params.dig(:position)
+    doc = @documents.dig(uri)
+    return session.reply(id, code: Code::INVALID_REQUEST, message: "Document #{uri} not found") unless doc
+    return session.reply(id, code: Code::INVALID_PARAMS, message: "Invalid position") unless position
+
+    # 防抖：取消同一文档上的旧补全请求，只处理最新请求
+    if (old_worker = @pending_completions.delete(uri))
+      old_worker.stop
+      logger.debug("Debounce: cancelled previous completion for #{uri}")
+    end
+    @pending_completions[uri] = Async::Task.current
+
+    # 延时等待，期间若有新请求会 stop 当前 task
+    # sleep 在 Async 中是非阻塞的，只暂停当前 task
+    sleep(@debounce_delay)
+
+    line = position.dig(:line)
+    char = position.dig(:character)
+
+    provider = current_provider
+    tokenizer = load_tokenizer(provider[:tokenizer_config])
+    prompt_data = build_prompt(doc, line, char, tokenizer, provider[:context_window], provider[:fim])
+    logger.debug("prompt: #{prompt_data[:prompt]&.length} chars, suffix: #{prompt_data[:suffix]&.length} chars")
+
+    items = llm_completions(prompt_data, provider[:tokens_to_clear]).map do |item|
+      {
+        insertText: item.fetch(:text, ""),
+        # 补充遥感信息
+        command: {
+          title: "Accept Completion",
+          tooltip: "accept inline completion",
+          command: "inlineCompletion/accept",
+          # arguments 允许任意类型数组 Any[]
+          arguments: ["llm-lsp", {
+            id: id,     # 复用消息 id
+            uri: uri,
+            position: position,
+            insertText: item[:text],
+            model: "#{item[:provider]}:#{item[:model]}",
+          }],
+        }
+      }
+    end
+    session.reply(id, result: { items: items })
+    track_completion(id) unless items.empty?
+  rescue Async::Stop
+    logger.debug("abort request: #{id}")
+  ensure
+    @pending_completions.delete(uri)
+  end
+
+  def llm_completions(prompt_data, tokens_to_clear)
+    logger.info("Calling LLM with provider #{@provider} ...")
+    chunks = []
+    params = {
+      model: current_provider[:model],
+      prompt: prompt_data[:prompt],
+      max_tokens: 500,
+      temperature: 0.01,
+      top_p: 0.9,
+      stop: [
+        # "\n", for line completion
+        "\n\n",
+      ],
+      stream: proc { |chunk, _event|
+        text = chunk.dig("choices", 0, "text")
+        chunks << text if text
+      },
+    }
+    params[:suffix] = prompt_data[:suffix] if prompt_data[:suffix]
+
+    current_client.completions(parameters: params)
+
+    result = chunks.join
+    tokens_to_clear.each { |tok| result.gsub!(tok, "") }
+    logger.info("LLM completion: #{result.length} chars")
+    logger.debug("LLM completion: #{result}")
+    return [] if result.empty?
+    [{
+      text: result,
+      provider: current_provider[:name],
+      model: current_provider[:model],
+    }]
+  end
+end

data/lib/lsp/code.rb

@@ -0,0 +1,9 @@
+module Code
+  PARSE_ERROR = -32700
+  INVALID_REQUEST = -32600
+  METHOD_NOT_FOUND = -32601
+  INVALID_PARAMS = -32602
+  INTERNAL_ERROR = -32603
+  SERVER_NOT_INITIALIZED = -32002
+  UNKNOWN_ERROR_CODE = -32001
+end

data/lib/lsp/json_rpc.rb

@@ -0,0 +1,55 @@
+require "json"
+require_relative "code"
+
+class JsonRpc
+  attr_reader :io
+
+  def initialize(io)
+    @io = io
+  end
+
+  def build_request(method, params: nil, id: nil)
+    j = {
+      jsonrpc: "2.0",
+      method: method,
+    }
+    j[:params] = params if params
+    j[:id] = id if id
+    j
+  end
+
+  def build_response(id, result: nil, code: nil, message: nil)
+    j = {
+      jsonrpc: "2.0",
+      id: id,
+    }
+    if result || (code.nil? && message.nil?)
+      j[:result] = result
+    else
+      j[:error] = {
+        code: code || Code::INVALID_REQUEST,
+        message: message || "",
+      }
+    end
+    j
+  end
+
+  def receive_message
+    headers = {}
+    while (line = io.gets) && line != "\r\n"
+      k, v = line.rstrip.split(/:\s*/)
+      headers[k.downcase] = v
+    end
+    length = headers.dig("content-length")&.to_i
+    return nil unless length
+    raw_json = io.read_exactly(length)
+    JSON.parse(raw_json, symbolize_names: true)
+  end
+
+  def send_message(message)
+    jsonrpc = JSON.generate(message)
+    body = "Content-Length: #{jsonrpc.bytesize}\r\n\r\n#{jsonrpc}"
+    io.write(body)
+    io.flush
+  end
+end

data/lib/lsp/popen.rb

@@ -0,0 +1,33 @@
+require "open3"
+require "io/stream"
+
+class Popen
+  attr_reader :input, :output, :wait_thr
+
+  def initialize(cmd)
+    pin, pout, @wait_thr = Open3.popen2(cmd)
+    @input = IO::Stream(pout)
+    @output = IO::Stream(pin)
+  end
+
+  def gets = input.gets
+  def read(size) = input.read(size)
+  def read_exactly(size) = input.read_exactly(size)
+  def write(buf) = output.write(buf)
+  def flush = output.flush
+
+  def close
+    input.close rescue nil
+    output.close rescue nil
+  end
+
+  def pid = wait_thr.pid
+
+  def alive? = wait_thr.alive?
+
+  def join(timeout = nil) = wait_thr.join(timeout)
+
+  def kill(signal = "TERM")
+    Process.kill(signal, pid) rescue nil
+  end
+end

data/lib/lsp/session.rb

@@ -0,0 +1,141 @@
+require "async"
+require_relative "code"
+
+class LspSession
+  attr_reader :name
+  attr_reader :channel
+  attr_reader :logger
+  attr :id
+
+  def initialize(name, channel, logger)
+    @name = name
+    @channel = channel
+    @logger = logger
+
+    @id = Random.rand(10)
+    @once_handlers = {}
+    @on_handlers = {}
+    @running_request = {}
+    # :setup, :initialized, :running, :shutdown, :exit
+    @state = :setup
+  end
+
+  def next_id
+    @id += 1
+  end
+
+  def start
+    Sync do |task|
+      # try call setup handler first
+      task.async do
+        @once_handlers.delete(:setup)&.call(nil)
+      rescue => e
+        logger.fatal(e)
+      end
+
+      # main loop
+      loop do
+        message = receive_message
+        next unless message
+
+        key ||= [:req, message[:method]] if message.key? :method
+        key ||= [:rsp, message[:id]] if message.key? :id
+        next logger.warn("#{name}: Invalid message #{message}, ignore") unless key
+
+        # speicial request
+        next if handle_special_request(message)
+        # normal request
+        worker = task.async do
+          h ||= @once_handlers.delete(key) || @once_handlers.delete(:*)
+          h ||= @on_handlers.dig(key) || @on_handlers.dig(:*)
+          next logger.debug("#{name}: no handler, ignore message #{message}") unless h
+          h.call(message)
+        rescue => e
+          raise
+        ensure
+          @running_request.delete(message[:id]) if message[:id]
+        end
+        if key.first == :req && id = message.dig(:id)
+          @running_request[id] = worker
+        end
+      # TODO: 规范异常处理
+      rescue => e
+        logger.fatal(e)
+      end
+    end
+  end
+
+  def handle_special_request(message)
+    method = message.dig(:method)
+    if "$/cancelRequest" == method
+      id = message.dig(:params, :id)
+      worker = @running_request.delete(id)
+      worker&.stop   # try stop it (send Async::Stop exception)
+      method
+    elsif "shutdown" == method
+      @state = :shutdown
+      reply(message[:id], result: nil)
+      method
+    elsif "exit" == method
+      code = @state == :shutdown ? 0 : 1
+      exit(code)
+    elsif [:shutdown, :exit].include?(@state)
+      # wait quitting, reject new request
+      reply(message.dig(:id), code: Code::INVALID_REQUEST, message: "Server is shutting down")
+      method
+    else
+      nil
+    end
+  end
+
+  def on(event=nil, &block)
+    event ||= :*
+    handlers = (event == :setup ? @once_handlers : @on_handlers)
+    handlers[event] = lambda { |m| block&.call(m) }
+  end
+  def on_method(method, &block) = on([:req, method], &block)
+
+  def once(event=nil, &block)
+    event ||= :*
+    @once_handlers[event] = lambda { |m| block&.call(m) }
+  end
+  def once_method(method, &block) = on([:req, method], &block)
+
+  def wait(event=nil, &block)
+    event ||= :*
+    cond = Async::Condition.new
+    @once_handlers[event] = lambda { |m| cond.signal(m) }
+    msg = cond.wait
+    block ? block.call(msg) : msg
+  end
+  def wait_method(method, &block) = wait([:req, method], &block)
+  def wait_response(id, &block) = wait([:rsp, id], &block)
+
+  def request(method, params: nil, id: nil)
+    id = next_id if id.nil?
+    message = channel.build_request(method, params:, id:)
+    send_message(message)
+    wait_response(id)
+  end
+
+  def notify(method, params: nil)
+    message = channel.build_request(method, params:)
+    send_message(message)
+  end
+
+  def reply(id, result: nil, code: nil, message: nil)
+    msg = channel.build_response(id, result:, code:, message:)
+    send_message(msg)
+  end
+
+  def send_message(msg)
+    channel.send_message(msg)
+    logger.debug { "<= #{msg}" }
+  end
+  
+  def receive_message
+    msg = channel.receive_message
+    logger.debug { "=> #{msg}" }
+    msg
+  end
+end

data/lib/lsp/stdio.rb

@@ -0,0 +1,25 @@
+require "io/stream"
+
+class Stdio
+  attr_reader :input
+  attr_reader :output
+
+  def initialize
+    # IO::Stream::Generic   对 IO 对象的基础封装，使其能更好的配合 Async
+    # IO::Stream::Buffered  继承 Generic 提供缓冲能力，并且支持 read_exactly, read_until 上层函数
+    # IO::Stream            module，并且重载了 call 能直接构造 Buffered 对象
+    @input = IO::Stream(STDIN)
+    @output = IO::Stream(STDOUT)
+  end
+
+  def gets = input.gets
+  def read(size) = input.read(size)
+  def read_exactly(size) = input.read_exactly(size)
+  def write(buf) = output.write(buf)
+  def flush = output.flush
+
+  def close
+    input.close
+    output.close
+  end
+end

metadata

@@ -0,0 +1,107 @@
+--- !ruby/object:Gem::Specification
+name: llm-lsp
+version: !ruby/object:Gem::Version
+  version: 0.2.0
+platform: ruby
+authors:
+- alpha0x00
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: async
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: io-stream
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.11'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.11'
+- !ruby/object:Gem::Dependency
+  name: ruby-openai
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '8.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '8.0'
+- !ruby/object:Gem::Dependency
+  name: tokenizers
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.6'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.6'
+description: Ruby LSP server that provides code completion using LLM FIM (Fill-In-the-Middle).
+  Supports Ollama and other OpenAI API compatible backends.
+executables:
+- llm-lsp
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- bin/llm-lsp
+- lib/core_ext/hash_deep_merge.rb
+- lib/llm_lsp.rb
+- lib/llm_lsp/version.rb
+- lib/llm_server.rb
+- lib/lsp/code.rb
+- lib/lsp/json_rpc.rb
+- lib/lsp/popen.rb
+- lib/lsp/session.rb
+- lib/lsp/stdio.rb
+homepage: https://github.com/leetking/llm-lsp.rb
+licenses:
+- MIT
+metadata: {}
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.1'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.6.9
+specification_version: 4
+summary: LLM-powered LSP server for code completion
+test_files: []