kward 0.66.0

data/doc/usage.md

@@ -0,0 +1,179 @@
+# Usage
+
+Kward's most common commands are:
+
+```bash
+kward                          # interactive chat
+kward "Explain this project"   # one-shot prompt
+kward --install-starter-pack   # optional first-time setup
+kward rpc                      # experimental JSON-RPC backend
+```
+
+When running from source, replace `kward` with `ruby lib/main.rb`.
+
+## Starter pack
+
+Install Kward's starter prompts and base `AGENTS.md` into your config directory, usually `~/.kward`:
+
+```bash
+kward --install-starter-pack
+```
+
+The installer downloads the pinned `kaiwood/kward-starter-pack` `v1.0.0` release, creates the config directory and base `config.json` if needed, and copies only starter-pack instruction/prompt files. It preserves the starter-pack layout in your config directory and skips files that already exist.
+
+## Interactive chat
+
+Interactive mode opens a terminal composer and saves the conversation as a per-workspace session. Use it when you want Kward to inspect files, make changes, run commands, or continue a conversation over time.
+
+The composer stays available while assistant and tool output streams. If you press Enter while a response is still running, Kward queues your next prompt and sends it after the current turn finishes. Press Ctrl+C while a response is running to stop the current turn and return to the composer.
+
+## One-shot prompts
+
+Pass a prompt as command-line text to ask once and exit:
+
+```bash
+kward "Summarize the changes in this repository"
+```
+
+Kward also accepts piped input:
+
+```bash
+git diff | kward "Review this diff"
+```
+
+One-shot prompts do not use Kward memory.
+
+## Workspace tools
+
+Kward can use these tools during a turn:
+
+- `list_directory` and `read_file` to inspect the workspace.
+- `write_file` and `edit_file` to create or change files.
+- `run_shell_command` to run confirmed local commands.
+- `web_search` to search the live web.
+- `code_search` to find packages, clone public GitHub repositories into cache, and read bounded source snippets.
+- `ask_user_question` to ask structured clarification questions.
+
+Safety rules:
+
+- Existing files must be read in the current conversation before Kward can write or edit them.
+- Every write, edit, and shell command asks for confirmation first.
+- Text file reads and edits are capped at 256 KiB per file.
+- When successful tool results include unified diffs, the composer status shows live session totals such as `+700|-572`.
+
+## Slash commands
+
+Use slash commands in interactive mode for local Kward actions:
+
+| Command | Purpose |
+| --- | --- |
+| `/exit`, `/quit` | Leave the session. |
+| `/new` | Start a fresh session. |
+| `/resume [path]` | Resume a saved session, or pick one when no path is given. |
+| `/name [name]` | Name or clear the current session name. |
+| `/clone` | Duplicate the current session. |
+| `/copy [last\|transcript]` | Copy clean assistant text or the Markdown transcript to the clipboard. |
+| `/export [path]` | Export the transcript as Markdown. |
+| `/compact [instructions]` | Summarize older conversation into a checkpoint and keep recent context. |
+| `/model` | Choose or type the default model. |
+| `/openrouter/catalog` | List the full OpenRouter model catalog. |
+| `/reasoning` | Choose reasoning effort. |
+| `/settings` | Configure overlay alignment and width. |
+| `/login` | Log in from inside the session. |
+| `/status` | Show status and auto-compaction information. |
+| `/stats [range]` | Show local telemetry stats when logging is enabled. |
+| `/memory ...` | Manage opt-in memory. |
+| `/redraw` | Refresh the terminal after resize or drawing glitches. |
+
+Prompt templates can add more slash commands. Plugin commands can also appear when trusted local plugins are installed.
+
+## Sessions
+
+Interactive chats are stored as JSONL files under `~/.kward/sessions/`. Sessions are per workspace.
+
+Useful session commands:
+
+- `/resume` opens a picker for recent sessions.
+- `/name <name>` gives the current session a human-readable name.
+- `/clone` creates a new independent copy. Cloned sessions remember their parent and appear indented in the resume picker.
+- `/copy` and `/copy last` copy the latest assistant response without terminal borders or ANSI styling. `/copy transcript` copies the clean Markdown transcript. Mouse selection may still include terminal UI chrome.
+- `/export [path]` writes a Markdown transcript. Explicit paths are resolved relative to the current workspace and must stay inside the workspace or Kward session directory.
+- `/compact [instructions]` summarizes older conversation into a structured Ruby-aware checkpoint. Text after `/compact ` is freeform focus text, not parsed as flags.
+
+After compaction, Kward may need to re-read files before future edits because compacting clears remembered read-file state.
+
+## Auto-compaction
+
+Auto-compaction is enabled by default when Kward can determine the active context window. It reserves part of the model context so conversations can continue longer without suddenly exceeding the limit.
+
+Configure it in `config.json`:
+
+```json
+{
+  "compaction": {
+    "enabled": true,
+    "reserve_tokens": 16384,
+    "keep_recent_tokens": 20000
+  }
+}
+```
+
+Manual `/compact` still works when auto-compaction is disabled.
+
+## Memory
+
+Memory is disabled by default and only applies to interactive sessions.
+
+Common commands:
+
+```text
+/memory enable
+/memory core <text>
+/memory add <text>
+/memory list
+/memory why
+/memory forget <id>
+/memory disable
+```
+
+See [Memory](memory.md) for storage locations, safety rules, auto-summary, and RPC methods.
+
+## Composer keys
+
+- Enter sends.
+- Shift+Enter inserts a newline.
+- Up/Down browse prompt history.
+- Ctrl+D exits from an empty prompt.
+- Ctrl+C stops the current response while assistant or tool output is running.
+- Backspace at the beginning of the prompt removes the most recent image attachment.
+
+Multiline input grows the composer up to a capped height.
+
+## Image attachments
+
+Kward can attach images when the active model supports images. It detects:
+
+- pasted image file paths,
+- Markdown image links,
+- `file://` image URLs,
+- image data URLs.
+
+In the interactive composer, pasted image references appear as attachment badges instead of editable filename or base64 text. Transcripts show attachment badges and render the image inline when the terminal advertises a supported inline image protocol, such as iTerm2 or Kitty/WezTerm.
+
+## Pan mode
+
+Pan mode starts a minimal LAN web UI with a prompt textarea and transcript:
+
+```bash
+kward --pan-mode --working-directory="/path/to/workspace"
+```
+
+From source:
+
+```bash
+ruby lib/main.rb --pan-mode --working-directory="/path/to/workspace"
+```
+
+Pan mode streams assistant output and tool calls, queues prompts submitted while a turn is running, and saves the conversation as a normal per-workspace session.
+
+Configure `pan_mode.username` and `pan_mode.password` before starting it; see [Configuration](configuration.md). Pan mode is reachable on the LAN, so use it only on trusted networks.

data/doc/web-search.md

@@ -0,0 +1,28 @@
+# Web search
+
+Web search lets the agent search the live web. Use it when you need current facts, official docs, release notes, bug reports, pricing pages, or recent announcements.
+
+Example prompts:
+
+```text
+Research the current Rails release notes and summarize migration risks.
+Find the official OpenRouter docs for model configuration.
+Check whether this dependency has a recent security advisory.
+```
+
+The `web_search` tool is advertised by default so the agent can use current sources when needed. In `auto` mode the provider fallback order is:
+
+1. Exa API when `EXA_API_KEY` is configured, otherwise keyless Exa MCP (`https://mcp.exa.ai/mcp`)
+2. Perplexity API when configured and `allow_model_providers` is true
+3. Gemini API with Google Search grounding when configured and `allow_model_providers` is true
+4. Legacy DuckDuckGo HTML search, then bundled public SearXNG instances
+
+Queries are sent over the network to the selected provider. API keys are never bundled with Kward; configure your own keys only if you want higher limits or alternate providers. Set `web_search.enabled` to `false` to hide the tool. Direct `provider: perplexity` or `provider: gemini` requests still use those providers when keys are configured.
+
+Supported arguments:
+
+- `queries`: one to four search strings
+- `max_results`: results per query, default 5, capped at 20
+- `provider`: optional `auto`, `exa`, `perplexity`, `gemini`, `legacy`, or `duckduckgo`
+- `recency_filter`: optional `day`, `week`, `month`, or `year`
+- `domain_filter`: optional list of included domains, or excluded domains prefixed with `-`

data/exe/kward

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require "kward"
+
+Kward::CLI.new.run

data/kward.gemspec

@@ -0,0 +1,33 @@
+require_relative "lib/kward/version"
+
+Gem::Specification.new do |spec|
+  spec.name = "kward"
+  spec.version = Kward::VERSION
+  spec.authors = ["Kai Wood"]
+  spec.email = ["kai.wood@icloud.com"]
+
+  spec.summary = "An extendable Ruby CLI coding agent."
+  spec.description = "Kward is a Ruby CLI coding agent with local workspace tools, configurable prompts, web search, sessions, and an experimental JSON-RPC backend."
+  spec.homepage = "https://github.com/kaiwood/kward"
+  spec.license = "MIT"
+  spec.required_ruby_version = ">= 3.2"
+
+  spec.metadata["rubygems_mfa_required"] = "true"
+
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |file|
+      file.start_with?(".ruby-lsp/", "test/", "plan/") || [".gitignore", "AGENTS.md"].include?(file)
+    end
+  end
+  spec.bindir = "exe"
+  spec.executables = ["kward"]
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "base64"
+  spec.add_dependency "nokogiri"
+  spec.add_dependency "tiktoken_ruby"
+  spec.add_dependency "tty-cursor"
+  spec.add_dependency "tty-prompt"
+  spec.add_dependency "tty-reader"
+  spec.add_dependency "tty-screen"
+end

data/lib/kward/agent.rb

@@ -0,0 +1,234 @@
+require_relative "cancellation"
+require_relative "model/chat_invocation"
+require_relative "compactor"
+require_relative "model/context_overflow"
+require_relative "conversation"
+require_relative "events"
+require_relative "steering"
+require_relative "telemetry/logger"
+require_relative "tools/registry"
+
+module Kward
+  # Runs model turns, handles context compaction, dispatches tool calls, and
+  # streams high-level events back to CLI and RPC callers.
+  class Agent
+    def initialize(client:, tool_registry: ToolRegistry.new, conversation: Conversation.new, telemetry_logger: TelemetryLogger.new)
+      @client = client
+      @tool_registry = tool_registry
+      @conversation = conversation
+      @telemetry_logger = telemetry_logger
+    end
+
+    attr_reader :conversation
+
+    # Adds a user message, compacts context when needed, and runs the turn.
+    #
+    # @param input [String] text sent to the model
+    # @param display_input [String, nil] alternate text kept for transcripts
+    # @yieldparam event [Object] streamed turn event for frontends
+    # @return [String] final assistant answer
+    def ask(input, display_input: nil, on_reasoning_delta: nil, on_retry: nil, cancellation: nil, steering: nil, &block)
+      started_at = @telemetry_logger.monotonic_now
+      status = "completed"
+      error = nil
+      cancellation&.raise_if_cancelled!
+      @conversation.refresh_system_message_if_workspace_agents_changed!
+      @conversation.append_user(input, display_content: display_input)
+      auto_compact_if_needed
+      run_turn(on_reasoning_delta: on_reasoning_delta, on_retry: on_retry, cancellation: cancellation, steering: steering, &block)
+    rescue StandardError => e
+      status = "failed"
+      error = e
+      raise e
+    ensure
+      log_turn(duration_ms: @telemetry_logger.duration_ms(started_at), status: status, error: error)
+    end
+
+    # Runs model calls until the assistant returns an answer without pending
+    # tool calls, including tool dispatch and one context-overflow retry.
+    #
+    # @yieldparam event [Object] streamed turn event for frontends
+    # @return [String] final assistant answer
+    def run_turn(on_reasoning_delta: nil, on_retry: nil, cancellation: nil, steering: nil)
+      overflow_retried = false
+      steering_state = build_steering_state(steering) do |event|
+        yield event if block_given?
+      end
+      loop do
+        cancellation&.raise_if_cancelled!
+        begin
+          message = chat(on_reasoning_delta: on_reasoning_delta, on_retry: on_retry, cancellation: cancellation, steering: steering) do |event|
+            yield event if block_given?
+          end
+        rescue StandardError => e
+          raise if cancellation&.cancelled?
+          raise unless !overflow_retried && ContextOverflow.error?(e) && compact_after_context_overflow(e)
+
+          overflow_retried = true
+          next
+        end
+        yield Events::AssistantMessage.new(message: message) if block_given?
+        @conversation.append_assistant(message)
+        steered_after_message = append_steering_events(steering_state)
+        yield Events::SteeringApplied.new(count: steered_after_message) if block_given? && steered_after_message.positive?
+
+        tool_calls = message["tool_calls"] || message[:tool_calls] || []
+        if tool_calls.empty?
+          next if steered_after_message.positive?
+
+          answer = safe_answer(message.fetch("content", message[:content] || ""))
+          yield Events::Answer.new(content: answer) if block_given?
+          return answer
+        end
+
+        tool_calls.each do |tool_call|
+          cancellation&.raise_if_cancelled!
+          yield Events::ToolCall.new(tool_call: tool_call) if block_given?
+          tool_started_at = @telemetry_logger.monotonic_now
+          content = nil
+          status = "completed"
+          error = nil
+          begin
+            content = @tool_registry.dispatch(tool_call, @conversation, cancellation: cancellation)
+          rescue StandardError => e
+            status = "failed"
+            error = e
+            raise e
+          ensure
+            log_tool(tool_call, content: content, duration_ms: @telemetry_logger.duration_ms(tool_started_at), status: status, error: error)
+          end
+          cancellation&.raise_if_cancelled!
+          yield Events::ToolResult.new(tool_call: tool_call, content: content) if block_given?
+        end
+        steered_after_tools = append_steering_events(steering_state)
+        yield Events::SteeringApplied.new(count: steered_after_tools) if block_given? && steered_after_tools.positive?
+      end
+    ensure
+      steering_state&.fetch(:unsubscribe)&.call
+    end
+
+    private
+
+    def build_steering_state(steering)
+      return nil unless steering
+
+      state = { events: [], appended: 0, mutex: Mutex.new, unsubscribe: nil }
+      state[:unsubscribe] = steering.on_submit do |steering_event|
+        state[:mutex].synchronize { state[:events] << steering_event }
+        yield Events::Steering.new(input: steering_event.input, created_at: steering_event.created_at)
+      end
+      state
+    end
+
+    def append_steering_events(state)
+      return 0 unless state
+
+      events = state[:mutex].synchronize do
+        state[:events][state[:appended]..] || []
+      end
+      events.each do |event|
+        @conversation.append_user(event.input)
+      end
+      state[:mutex].synchronize { state[:appended] += events.length }
+      events.length
+    end
+
+    def log_turn(duration_ms:, status:, error:)
+      payload = { "duration_ms" => duration_ms, "status" => status }
+      @telemetry_logger.log("performance", "turn", payload)
+      log_error("turn_error", error, payload) if error
+    end
+
+    def log_tool(tool_call, content:, duration_ms:, status:, error:)
+      payload = {
+        "tool_name" => tool_name(tool_call),
+        "duration_ms" => duration_ms,
+        "status" => status,
+        "result_bytes" => content.to_s.bytesize
+      }
+      @telemetry_logger.log("tools", "tool_call", payload)
+      @telemetry_logger.log("performance", "tool_call", payload)
+      log_error("tool_error", error, payload) if error
+    end
+
+    def log_error(event, error, payload = {})
+      return unless error
+
+      @telemetry_logger.log("errors", event, payload.merge(TelemetryLogger.error_payload(error)))
+    end
+
+    def tool_name(tool_call)
+      function = tool_call["function"] || tool_call[:function] || {}
+      function["name"] || function[:name]
+    end
+
+    def auto_compact_if_needed
+      context_window = @client.current_context_window if @client.respond_to?(:current_context_window)
+      Compactor.new(conversation: @conversation, client: @client).auto_compact_if_needed(context_window: context_window)
+    rescue StandardError => e
+      warn "Auto-compaction failed: #{e.message}"
+      nil
+    end
+
+    def compact_after_context_overflow(error)
+      settings = Compaction::Settings.from_config
+      return nil unless settings.enabled
+
+      Compactor.new(conversation: @conversation, client: @client, settings: settings).compact(
+        custom_instructions: "The previous model request exceeded the context window. Preserve the current task state and critical details needed to retry."
+      )
+    rescue Compaction::NothingToCompact, Compaction::AlreadyCompacted, StandardError => compaction_error
+      warn "Context overflow recovery failed: #{compaction_error.message}; original error: #{error.message}"
+      nil
+    end
+
+    def chat(on_reasoning_delta: nil, on_retry: nil, cancellation: nil, steering: nil)
+      reasoning_delta = lambda do |delta|
+        cancellation&.raise_if_cancelled!
+        on_reasoning_delta&.call(delta)
+        yield Events::ReasoningDelta.new(delta: delta) if block_given?
+      end
+      assistant_delta = lambda do |delta|
+        cancellation&.raise_if_cancelled!
+        yield Events::AssistantDelta.new(delta: delta) if block_given?
+      end
+      retry_callback = lambda do |retry_info|
+        cancellation&.raise_if_cancelled!
+        event = Events::Retry.new(**retry_info)
+        on_retry&.call(event)
+        yield event if block_given?
+      end
+      ChatInvocation.call(
+        @client,
+        @conversation.messages,
+        {
+          tools: @tool_registry.schemas,
+          on_reasoning_delta: reasoning_delta,
+          on_assistant_delta: assistant_delta,
+          on_retry: retry_callback,
+          cancellation: cancellation,
+          steering: steering
+        }
+      )
+    end
+
+    def safe_answer(content)
+      text = content.to_s
+      return text unless claims_file_edit?(text)
+      return text if last_file_change_succeeded?
+      return "The file change tool returned an error or declined result, so I did not successfully change the file." if @conversation.last_file_change_result
+
+      "I have not changed any files. I need to use write_file or edit_file successfully before claiming a file change."
+    end
+
+    def claims_file_edit?(text)
+      text.match?(/\b(I|I've|I have)\s+(changed|updated|modified|edited|created|deleted|wrote)\b/i)
+    end
+
+    def last_file_change_succeeded?
+      result = @conversation.last_file_change_result
+      content = result&.fetch(:content, nil) || result&.fetch("content", "")
+      content&.start_with?("Wrote ", "Edited ")
+    end
+  end
+end