elelem 0.9.1 → 0.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ab9d6c213098f0a206e17684325c3f756ee1456ec05040eb8b2e96b46c9e5ee5
-  data.tar.gz: a6db8b35f17c43558a799198785fef34e062c861a73b000eb148531fe6a1f2dc
+  metadata.gz: 3e95bbe823f4cf39a8a801bd32240caa47dcc3bdea6ce803f7c744f5f192f74c
+  data.tar.gz: 685b6b52ff14408979bd61a0ea42e87d24602f398392ac845fcc893eceabccae
 SHA512:
-  metadata.gz: eb916323872221ee6b8a6233a078f59183f91c082fc926144e1d91fe36176bc34645cc613b30372ac89e5668697dcf99ad36624cf7aa80d17076a911e7f6d78a
-  data.tar.gz: e7d619049618bc475c1f143d9e6fe46df2a58a5288fb916da790ae1ecde2cd6c0f992f04e95c2f365faaa17424d20ed8cc214d10a1f4b9081d54c13481966331
+  metadata.gz: cc605e03c30962a7f75d0846cde1bfb46d41c73ffbd07ee49f933fe51076ae08df7f9fb86b933d16864fd5aad7de9a29297d3b7da01337a9aa62d80aa9fa5940
+  data.tar.gz: 5d2580db0a5e607c360bb1f691d2eabaec7a37af7ca4f705950fed4048a567be46077c04e5a289163473f679ec95b76b8b04e5c56d84a3bb4eb42be6cc1b4da6

data/CHANGELOG.md

@@ -1,3 +1,37 @@
+## [0.10.0] - 2026-01-27
+
+### Added
+- **Async MCP loading** for faster startup - tools load in background thread
+- **HTTP MCP servers** with SSE support and session management
+- **OAuth authentication** for MCP servers with PKCE, automatic token refresh
+- **Global hooks** - `toolbox.before`/`toolbox.after` without tool name applies to all tools
+- **`/context` improvements**: `/context <n>` to view entry, `/context json` for full dump
+- **ast-grep (`sg`) support** for building repo maps - faster and more accurate than ctags
+- **New tools**: `glob`, `grep`, `list`, `git`, `task`, `/tools` command
+- **Permissions system** (`lib/elelem/permissions.rb`) for tool access control
+- **OpenAI reasoning mode** - enables `Reasoning: high` for o-series models
+- **Test coverage** for OAuth, token storage, HTTP MCP, SSE parsing, global hooks
+
+### Changed
+- **BREAKING: Plugin API** - plugins now receive `agent` instead of `toolbox`
+  - Old: `Elelem::Plugins.register(:name) { |toolbox| toolbox.add(...) }`
+  - New: `Elelem::Plugins.register(:name) { |agent| agent.toolbox.add(...) }`
+  - Plugins can now access `agent.terminal`, `agent.commands`, `agent.conversation`
+- Extracted `Conversation` class from `Agent` for better separation of concerns
+- Extracted `Commands` class for slash command handling
+- Refactored LLM fetch interface to emit separate events for thinking/content/tool_calls
+- Simplified system prompt with inline ERB template
+- Renamed confirm plugin to `zz_confirm` to ensure it loads last
+- MCP logs now write to `~/.elelem/mcp.log` instead of working directory
+- Tool schema now frozen to prevent mutation
+- Uses `Open3.capture2` instead of backticks for thread safety
+- Improved ANSI escape sequence stripping in `/shell` transcripts
+
+## [0.9.2] - 2026-01-22
+
+### Fixed
+- Suppress pathname gem constant redefinition warnings on Ruby 4.0
+
 ## [0.9.1] - 2026-01-22
 
 ### Fixed

data/README.md

@@ -31,17 +31,18 @@ Elelem relies on several external tools. Install the ones you need:
 
 | Tool | Purpose | Install |
 |------|---------|---------|
-| [Ollama](https://ollama.ai/) | Default LLM provider | https://ollama.ai/download |
-| [glow](https://github.com/charmbracelet/glow) | Markdown rendering | `brew install glow` / `go install github.com/charmbracelet/glow@latest` |
+| [ast-grep](https://ast-grep.github.io/) | Structural search (`sg`) | `brew install ast-grep` / `cargo install ast-grep` |
 | [ctags](https://ctags.io/) | Repo map generation | `brew install universal-ctags` / `apt install universal-ctags` |
-| [ripgrep](https://github.com/BurntSushi/ripgrep) | Text search (`rg`) | `brew install ripgrep` / `apt install ripgrep` |
 | [fd](https://github.com/sharkdp/fd) | File discovery | `brew install fd` / `apt install fd-find` |
-| [ast-grep](https://ast-grep.github.io/) | Structural search (`sg`) | `brew install ast-grep` / `cargo install ast-grep` |
-| [Git](https://git-scm.com/) | Version control | `brew install git` / `apt install git` |
+| [git](https://git-scm.com/) | Version control | `brew install git` / `apt install git` |
+| [glow](https://github.com/charmbracelet/glow) | Markdown rendering | `brew install glow` / `go install github.com/charmbracelet/glow@latest` |
+| [jq](https://jqlang.github.io/jq/) | JSON processing | `brew install jq` / `apt install jq` |
+| [ollama](https://ollama.ai/) | Default LLM provider | https://ollama.ai/download |
+| [ripgrep](https://github.com/BurntSushi/ripgrep) | Text search (`rg`) | `brew install ripgrep` / `apt install ripgrep` |
 
 **Required:** Git, Ollama (or another LLM provider)
 
-**Recommended:** glow, ctags, ripgrep, fd
+**Recommended:** glow, jq, ctags, ripgrep, fd
 
 **Optional:** ast-grep (for structural code search)
 
@@ -128,18 +129,94 @@ Each provider reads its configuration from environment variables:
 * **Conversation History** – persists across turns; can be cleared.
 * **Context Dump** – `/context` shows the current conversation state.
 
-## Toolbox Overview
-
-The `Toolbox` class is defined in `lib/elelem/toolbox.rb`. It supplies
-three tools, each represented by a JSON schema that the LLM can call.
+## Tools
+
+Built-in tools available to the LLM:
+
+| Tool      | Purpose                    | Parameters                |
+| --------- | -------------------------- | ------------------------- |
+| `read`    | Read file contents         | `path`                    |
+| `write`   | Write file                 | `path`, `content`         |
+| `edit`    | Replace text in file       | `path`, `old`, `new`      |
+| `execute` | Run shell command          | `command`                 |
+| `eval`    | Execute Ruby code          | `ruby`                    |
+| `glob`    | Find files by pattern      | `pattern`, `path`         |
+| `grep`    | Search file contents       | `pattern`, `path`, `glob` |
+| `list`    | List directory             | `path`, `recursive`       |
+| `git`     | Run git command            | `command`, `args`         |
+| `task`    | Delegate to sub-agent      | `prompt`                  |
+| `verify`  | Check syntax and run tests | `path`                    |
+
+Aliases: `bash`, `sh`, `exec` → `execute`; `open` → `read`; `ls` → `list`
+
+## Plugins
+
+Plugins extend elelem with custom tools and commands. They are loaded from:
+- `lib/elelem/plugins/` (built-in)
+- `~/.elelem/plugins/` (user global)
+- `.elelem/plugins/` (project local)
+
+### Writing a Plugin
+
+```ruby
+# ~/.elelem/plugins/hello.rb
+Elelem::Plugins.register(:hello) do |agent|
+  # Add a tool
+  agent.toolbox.add("hello",
+    description: "Say hello",
+    params: { name: { type: "string" } },
+    required: ["name"]
+  ) do |args|
+    { message: "Hello, #{args["name"]}!" }
+  end
+
+  # Add a command
+  agent.commands.register("greet", description: "Greet the user") do
+    agent.terminal.say "Hello!"
+  end
+
+  # Add hooks
+  agent.toolbox.before("execute") { |args| puts "Running: #{args["command"]}" }
+  agent.toolbox.after("execute") { |args, result| puts "Exit: #{result[:exit_status]}" }
+
+  # Global hook (runs for all tools)
+  agent.toolbox.before { |args, tool_name:| puts "Calling #{tool_name}" }
+end
+```
 
-| Tool      | Purpose            | Parameters         |
-| --------- | ------------------ | ------------------ |
-| `read`    | Read file contents | `path`             |
-| `write`   | Write file         | `path`, `content`  |
-| `execute` | Run shell command  | `command`          |
+### Plugin API
+
+Plugins receive an `agent` object with access to:
+- `agent.toolbox` - add tools, register hooks
+- `agent.terminal` - output to the user (`say`, `ask`, `markdown`)
+- `agent.commands` - register slash commands
+- `agent.conversation` - access message history
+- `agent.client` - the LLM client
+- `agent.fork(system_prompt:)` - create a sub-agent
+
+## MCP Configuration
+
+Configure MCP servers in `~/.elelem/mcp.json` or `.elelem/mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "gitlab": {
+      "command": "npx",
+      "args": ["-y", "@anthropics/gitlab-mcp"],
+      "env": {
+        "GITLAB_TOKEN": "${GITLAB_TOKEN}"
+      }
+    },
+    "remote": {
+      "type": "http",
+      "url": "https://mcp.example.com/sse"
+    }
+  }
+}
+```
 
-Aliases: `bash`, `sh`, `exec` → `execute`; `open` → `read`
+HTTP servers support OAuth authentication automatically.
 
 ## Known Limitations
 

data/Rakefile

@@ -5,15 +5,4 @@ require "rspec/core/rake_task"
 
 RSpec::Core::RakeTask.new(:spec)
 
-task :files do
-  IO.popen(%w[git ls-files], chdir: __dir__, err: IO::NULL) do |ls|
-    ls.readlines.each do |f|
-      next if f.start_with?(*%w[bin/ spec/ pkg/ .git .rspec Gemfile Rakefile])
-      next if f.strip.end_with?(*%w[.toml .txt .md])
-
-      puts f
-    end
-  end
-end
-
 task default: %i[spec]

data/exe/elelem

@@ -1,85 +1,8 @@
-#!/usr/bin/env ruby
+#!/usr/bin/env -S ruby -W0
 # frozen_string_literal: true
 
 require "elelem"
-require "optparse"
 
 Signal.trap("INT") { exit 1 }
 
-class App
-  MODELS = {
-    "ollama" => "gpt-oss:latest",
-    "anthropic" => "claude-opus-4-5-20250514",
-    "vertex" => "claude-opus-4-5@20251101",
-    "openai" => "gpt-4o"
-  }.freeze
-
-  PROVIDERS = {
-    "ollama" => ->(model) { Elelem::Net::Ollama.new(model: model, host: ENV.fetch("OLLAMA_HOST", "localhost:11434")) },
-    "anthropic" => ->(model) { Elelem::Net::Claude.anthropic(model: model, api_key: ENV.fetch("ANTHROPIC_API_KEY")) },
-    "vertex" => ->(model) { Elelem::Net::Claude.vertex(model: model, project: ENV.fetch("GOOGLE_CLOUD_PROJECT"), region: ENV.fetch("GOOGLE_CLOUD_REGION", "us-east5")) },
-    "openai" => ->(model) { Elelem::Net::OpenAI.new(model: model, api_key: ENV.fetch("OPENAI_API_KEY")) }
-  }.freeze
-
-  def initialize(args)
-    @provider = "ollama"
-    @model = nil
-    @args = parse(args)
-  end
-
-  def run
-    command = @args.shift || "chat"
-    send(command.tr("-", "_"))
-  rescue NoMethodError
-    abort "Unknown command: #{command}"
-  end
-
-  private
-
-  def parse(args)
-    @parser = OptionParser.new do |o|
-      o.banner = "Usage: elelem [command] [options] [args]"
-      o.separator "\nCommands:"
-      o.separator "  chat              Interactive REPL (default)"
-      o.separator "  ask <prompt>      One-shot query (reads stdin if piped)"
-      o.separator "  files             Output files as XML (no options)"
-      o.separator "  help              Show this help"
-      o.separator "\nOptions:"
-      o.on("-p", "--provider NAME", "ollama, anthropic, vertex, openai") { |p| @provider = p }
-      o.on("-m", "--model NAME", "Override default model") { |m| @model = m }
-      o.on("-h", "--help") { puts o; exit }
-    end
-    @parser.parse!(args)
-  end
-
-  def help
-    puts @parser
-  end
-
-  def client
-    model = @model || MODELS.fetch(@provider)
-    PROVIDERS.fetch(@provider).call(model)
-  end
-
-  def chat = Elelem.start(client)
-
-  def ask
-    abort "Usage: elelem ask <prompt>" if @args.empty?
-    prompt = @args.join(" ")
-    prompt = "#{prompt}\n\n```\n#{$stdin.read}\n```" if $stdin.stat.pipe?
-    Elelem::Terminal.new.markdown Elelem.ask(client, prompt)
-  end
-
-  def files
-    files = $stdin.stat.pipe? ? $stdin.readlines : `git ls-files`.lines
-    puts "<documents>"
-    files.each_with_index do |line, i|
-      path = line.strip
-      next if path.empty? || !File.file?(path)
-      puts %Q{<document index="#{i + 1}"><source>#{path}</source><content><![CDATA[#{File.read(path)}]]></content></document>}
-    end
-    puts "</documents>"
-  end
-end
-
-App.new(ARGV).run
+Elelem::CLI.new(ARGV).run

data/lib/elelem/agent.rb

@@ -2,40 +2,16 @@
 
 module Elelem
   class Agent
-    COMMANDS = %w[/clear /context /init /reload /shell /exit /help].freeze
-    MAX_CONTEXT_MESSAGES = 50
-    INIT_PROMPT = <<~PROMPT
-      AGENTS.md generator. Analyze codebase and write AGENTS.md to project root.
+    attr_reader :conversation, :client, :toolbox, :terminal, :commands
+    attr_writer :terminal, :toolbox, :commands
 
-      # AGENTS.md Spec (https://agents.md/)
-      A file providing context and instructions for AI coding agents.
-
-      ## Recommended Sections
-      - Commands: build, test, lint commands
-      - Code Style: conventions, patterns
-      - Architecture: key components and flow
-      - Testing: how to run tests
-
-      ## Process
-      1. Read README.md if present
-      2. Identify language (Gemfile, package.json, go.mod)
-      3. Find test scripts (bin/test, npm test)
-      4. Check linter configs
-      5. Write concise AGENTS.md
-
-      Keep it minimal. No fluff.
-    PROMPT
-
-    attr_reader :history, :client, :toolbox, :terminal
-
-    def initialize(client, toolbox, terminal: nil, history: nil, system_prompt: nil)
+    def initialize(client, toolbox: Toolbox.new, terminal: nil, system_prompt: nil, commands: nil)
       @client = client
       @toolbox = toolbox
-      @terminal = terminal || Terminal.new(commands: COMMANDS)
-      @history = history || []
+      @commands = commands || Commands.new
+      @terminal = terminal
+      @conversation = Conversation.new
       @system_prompt = system_prompt
-      @memory = nil
-      register_task_tool
     end
 
     def repl
@@ -49,27 +25,21 @@ module Elelem
     end
 
     def command(input)
-      case input
-      when "/exit" then exit(0)
-      when "/init" then init_agents_md
-      when "/reload" then reload_source!
-      when "/shell"
-        transcript = start_shell
-        history << { role: "user", content: transcript } unless transcript.strip.empty?
-      when "/clear"
-        @history = []
-        @memory = nil
-        terminal.say "  → context cleared"
-      when "/context"
-        terminal.say JSON.pretty_generate(combined_history)
-      else
-        terminal.say COMMANDS.join(" ")
-      end
+      parts = input.delete_prefix("/").split(" ", 2)
+      name, args = parts[0], parts[1]
+      commands.run(name, args) || terminal.say(commands.names.join(" "))
+    end
+
+    def context
+      @conversation.to_a(system_prompt: system_prompt)
+    end
+
+    def fork(system_prompt:)
+      Agent.new(client, toolbox: toolbox, terminal: terminal, system_prompt: system_prompt)
     end
 
     def turn(input)
-      compact_if_needed
-      history << { role: "user", content: input }
+      @conversation.add(role: "user", content: input)
       ctx = []
       content = nil
 
@@ -85,7 +55,7 @@ module Elelem
         end
       end
 
-      history << { role: "assistant", content: content }
+      @conversation.add(role: "assistant", content: content)
       content
     end
 
@@ -97,90 +67,29 @@ module Elelem
       toolbox.run(name.to_s, args)
     end
 
-    def register_task_tool
-      @toolbox.add("task",
-        description: "Delegate subtask to focused agent (complex searches, multi-file analysis)",
-        params: { prompt: { type: "string" } },
-        required: ["prompt"]
-      ) do |a|
-        sub = Agent.new(client, toolbox, terminal: terminal,
-          system_prompt: "Research agent. Search, analyze, report. Be concise.")
-        sub.turn(a["prompt"])
-        { result: sub.history.last[:content] }
-      end
-    end
-
-    def init_agents_md
-      sub = Agent.new(client, toolbox, terminal: terminal, system_prompt: INIT_PROMPT)
-      sub.turn("Generate AGENTS.md for this project")
-    end
-
-    def reload_source!
-      lib_dir = File.expand_path("..", __dir__)
-      original_verbose, $VERBOSE = $VERBOSE, nil
-      Dir["#{lib_dir}/**/*.rb"].sort.each { |f| load(f) }
-      $VERBOSE = original_verbose
-      @toolbox = Toolbox.new
-      Plugins.reload!(@toolbox)
-      register_task_tool
-    end
-
-    def start_shell
-      Tempfile.create do |file|
-        system("script", "-q", file.path, chdir: Dir.pwd)
-        strip_ansi(File.read(file.path))
-      end
-    end
-
-    def strip_ansi(text)
-      text.gsub(/^Script started.*?\n/, "")
-          .gsub(/\nScript done.*$/, "")
-          .gsub(/\e\[[0-9;]*[a-zA-Z]/, "")
-          .gsub(/\e\[\?[0-9]+[hl]/, "")
-          .gsub(/[\b]/, "")
-          .gsub(/\r/, "")
-    end
-
     def fetch_response(ctx)
-      content = ""
-      tool_calls = client.fetch(combined_history + ctx, toolbox.to_a) do |delta|
-        content += delta[:content].to_s
-        terminal.print(terminal.think(delta[:thinking])) if delta[:thinking]
+      content = String.new
+      tool_calls = []
+
+      client.fetch(@conversation.to_a(system_prompt: system_prompt) + ctx, toolbox.to_a) do |event|
+        case event[:type]
+        when "saying"
+          content << event[:text].to_s
+        when "thinking"
+          terminal.print(terminal.think(event[:text]))
+        when "tool_call"
+          tool_calls << { id: event[:id], name: event[:name], arguments: event[:arguments] }
+        end
       end
+
       [content, tool_calls]
     rescue => e
       terminal.say "\n  ✗ #{e.message}"
       ["Error: #{e.message} #{e.backtrace.join("\n")}", []]
     end
 
-    def combined_history
-      [{ role: "system", content: system_prompt }] + history
-    end
-
     def system_prompt
-      @system_prompt || SystemPrompt.new(memory: @memory).render
-    end
-
-    def compact_if_needed
-      return if history.length <= MAX_CONTEXT_MESSAGES
-
-      terminal.say "  → compacting context"
-      keep = MAX_CONTEXT_MESSAGES / 2
-      old = history.first(history.length - keep)
-
-      to_summarize = @memory ? [{ role: "memory", content: @memory }, *old] : old
-      @memory = summarize(to_summarize)
-      @history = history.last(keep)
-    end
-
-    def summarize(messages)
-      text = messages.map { |message| { role: message[:role], content: message[:content] } }.to_json
-
-      String.new.tap do |buffer|
-        client.fetch([{ role: "user", content: "Summarize key facts:\n#{text}" }], []) do |d|
-          buffer << d[:content].to_s
-        end
-      end
+      @system_prompt || SystemPrompt.new.render
     end
   end
 end

data/lib/elelem/commands.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+module Elelem
+  class Commands
+    def initialize
+      @registry = {}
+    end
+
+    def register(name, description: "", &handler)
+      @registry[name] = { description: description, handler: handler }
+    end
+
+    def run(name, args = nil)
+      entry = @registry[name]
+      return false unless entry
+
+      entry[:handler].arity == 0 ? entry[:handler].call : entry[:handler].call(args)
+      true
+    end
+
+    def names
+      @registry.keys.map { |name| "/#{name}" }
+    end
+
+    def each
+      @registry.each { |name, entry| yield "/#{name}", entry[:description] }
+    end
+
+    def include?(name)
+      @registry.key?(name)
+    end
+  end
+end

data/lib/elelem/conversation.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module Elelem
+  class Conversation
+    ROLES = %w[user assistant tool].freeze
+
+    def initialize
+      @messages = []
+    end
+
+    def add(role:, content:, **extra)
+      raise ArgumentError, "invalid role: #{role}" unless ROLES.include?(role)
+      @messages << { role: role, content: content, **extra }.compact
+    end
+
+    def last = @messages.last
+    def length = @messages.length
+    def clear! = @messages.clear
+
+    def to_a(system_prompt: nil)
+      base = system_prompt ? [{ role: "system", content: system_prompt }] : []
+      base + @messages
+    end
+  end
+end