kward 0.68.0 → 0.69.0

data/doc/web-search.md

@@ -1,28 +1,94 @@
 # Web search
 
-Web search lets the agent search the live web. Use it when you need current facts, official docs, release notes, bug reports, pricing pages, or recent announcements.
+Use web search when the answer depends on current or external information:
+
+- current framework or dependency docs,
+- release notes and migration guides,
+- security advisories,
+- pricing or provider pages,
+- bug reports and issue discussions,
+- a specific URL you want Kward to inspect.
 
 Example prompts:
 
 ```text
-Research the current Rails release notes and summarize migration risks.
+Check the current Rails release notes and summarize migration risks for this project.
 Find the official OpenRouter docs for model configuration.
 Check whether this dependency has a recent security advisory.
+Read this URL and explain the setup steps: https://example.com/docs
+```
+
+## How Kward researches
+
+Kward has three web tools:
+
+1. `web_search` finds candidate sources.
+2. `fetch_content` reads human-readable pages.
+3. `fetch_raw` reads machine-readable files such as JSON, YAML, XML, RSS, OpenAPI specs, or plain text.
+
+A good research flow is:
+
+```text
+Search for the official docs, fetch the relevant page, then answer with the source URL.
+```
+
+Kward should search first, then fetch important pages before relying on them.
+
+## Network behavior
+
+Web tools are advertised to the model by default. Queries and fetched URLs are sent over the network to the selected provider or target host.
+
+In automatic mode, provider fallback is:
+
+1. Exa API when `EXA_API_KEY` is configured, otherwise keyless Exa MCP.
+2. Perplexity API when configured and model-provider fallback is allowed.
+3. Gemini API with Google Search grounding when configured and model-provider fallback is allowed.
+4. DuckDuckGo HTML search, then bundled public SearXNG instances.
+
+You do not need an API key for basic web search, but keys can improve limits or provider choice.
+
+## Disable web tools
+
+Hide all web tools:
+
+```json
+{
+  "web_search": {
+    "enabled": false
+  }
+}
 ```
 
-The `web_search` tool is advertised by default so the agent can use current sources when needed. In `auto` mode the provider fallback order is:
+Use this when working on private projects where no prompt should trigger external lookup.
+
+## Tool details
+
+### `web_search`
+
+Finds candidate sources. Arguments:
+
+- `queries`: one to four search strings.
+- `max_results`: results per query, default 5, capped at 20.
+- `provider`: optional `auto`, `exa`, `perplexity`, `gemini`, or `duckduckgo`.
+- `recency_filter`: optional `day`, `week`, `month`, or `year`.
+- `domain_filter`: optional domains to include, or domains prefixed with `-` to exclude.
+
+### `fetch_content`
+
+Reads a specific HTTP or HTTPS page and extracts readable text. Use it for docs pages, articles, issues, and release notes.
+
+Arguments:
+
+- `url`
+- `max_bytes`: default 16384, capped at 131072.
+- `extract`: optional `auto`, `text`, or `markdown`.
 
-1. Exa API when `EXA_API_KEY` is configured, otherwise keyless Exa MCP (`https://mcp.exa.ai/mcp`)
-2. Perplexity API when configured and `allow_model_providers` is true
-3. Gemini API with Google Search grounding when configured and `allow_model_providers` is true
-4. DuckDuckGo HTML search, then bundled public SearXNG instances
+### `fetch_raw`
 
-Queries are sent over the network to the selected provider. API keys are never bundled with Kward; configure your own keys only if you want higher limits or alternate providers. Set `web_search.enabled` to `false` to hide the tool. Direct `provider: perplexity` or `provider: gemini` requests still use those providers when keys are configured.
+Reads a specific HTTP or HTTPS resource without readability extraction. Use it for JSON, YAML, XML, RSS, OpenAPI specs, and plain text.
 
-Supported arguments:
+Arguments:
 
-- `queries`: one to four search strings
-- `max_results`: results per query, default 5, capped at 20
-- `provider`: optional `auto`, `exa`, `perplexity`, `gemini`, or `duckduckgo`
-- `recency_filter`: optional `day`, `week`, `month`, or `year`
-- `domain_filter`: optional list of included domains, or excluded domains prefixed with `-`
+- `url`
+- `max_bytes`: default 16384, capped at 131072.
+- `accept`: optional HTTP `Accept` header.

data/exe/kward

@@ -1,5 +1,7 @@
 #!/usr/bin/env ruby
 
+$LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
+
 require "kward"
 
 Kward::CLI.new.run

data/lib/kward/agent.rb

@@ -213,7 +213,7 @@ module Kward
       end
       ChatInvocation.call(
         @client,
-        @conversation.messages,
+        @conversation.context_messages,
         {
           tools: @tool_registry.schemas,
           on_reasoning_delta: reasoning_delta,

data/lib/kward/cli/commands.rb

@@ -50,8 +50,9 @@ module Kward
             #{command.call("kward")} #{option.call('"Explain this project"')}       Run a one-shot prompt
             #{command.call("kward login")}                        Sign in or save provider credentials
             #{command.call("kward auth status")}                  Show saved credential status
-            #{command.call("kward init")}                         Install starter prompts and AGENTS.md
+            #{command.call("kward init")}                         Install starter prompts and PRINCIPLES.md
             #{command.call("kward doctor")}                       Check local Kward setup
+            #{command.call("kward sysprompt")}                    Inspect the effective system prompt
             #{command.call("kward pan")}                          Start Pan mode web UI
             #{command.call("kward rpc")}                          Start the experimental JSON-RPC backend
 
@@ -60,8 +61,9 @@ module Kward
             #{command.call("version")}                            Show the installed Kward version
             #{command.call("login")} [anthropic|openrouter|github] Sign in with OpenAI, Anthropic, OpenRouter, or GitHub
             #{command.call("auth status|logout")}                 Show or clear saved credentials
-            #{command.call("init")}                               Install starter prompts and AGENTS.md
+            #{command.call("init")}                               Install starter prompts and PRINCIPLES.md
             #{command.call("doctor")}                             Check local Kward setup
+            #{command.call("sysprompt")} [--raw]                  Inspect the effective system prompt
             #{command.call("stats tokens")} [range] [options]      Export local token telemetry as CSV
             #{command.call("pan")}                                Start Pan mode web UI
             #{command.call("rpc")}                                Run the JSON-RPC backend for UI clients
@@ -106,7 +108,7 @@ module Kward
           },
           "init" => {
             usage: "kward init",
-            description: "Install starter prompts and base AGENTS.md into your config directory.",
+            description: "Install starter prompts and base PRINCIPLES.md into your config directory.",
             examples: ["kward init"]
           },
           "doctor" => {
@@ -114,6 +116,11 @@ module Kward
             description: "Check local Kward configuration, workspace, auth hints, and writable directories.",
             examples: ["kward doctor", "kward --working-directory ~/code/project doctor"]
           },
+          "sysprompt" => {
+            usage: "kward sysprompt [--raw]",
+            description: "Inspect the effective system prompt for a new conversation in the current workspace.",
+            examples: ["kward sysprompt", "kward sysprompt --raw", "kward --working-directory ~/code/project sysprompt"]
+          },
           "stats" => {
             usage: "kward stats tokens [range] [--bucket second|minute|hour|day|week|month|year] [--output path]",
             description: "Export local token telemetry as CSV.",

data/lib/kward/cli/compaction.rb

@@ -12,12 +12,12 @@ module Kward
           client: @client,
           tool_result_summarizer: lambda { |tool_call, content| tool_result_summary(tool_call, content) }
         ).compact(custom_instructions: argument)
-        @prompt.say("\nCompacted context: #{result.old_message_count} messages -> #{result.new_message_count} messages.\n")
+        runtime_output("Compacted context: #{result.old_message_count} messages -> #{result.new_message_count} messages.")
         render_transcript_block("Assistant", result.summary)
       rescue Compactor::NothingToCompact, Compactor::AlreadyCompacted, Compactor::EmptySummary => e
-        @prompt.say("\n#{e.message}\n")
+        runtime_output(e.message)
       rescue StandardError => e
-        @prompt.say("\nCompaction error: #{e.message}\n")
+        runtime_output("Compaction error: #{e.message}")
       end
 
     end

data/lib/kward/cli/interactive_turn.rb

@@ -95,6 +95,9 @@ module Kward
         when Events::AssistantDelta
           stream_state[:streamed] = true
           append_markdown_delta(markdown_chunks, "Assistant", event.delta)
+        when Events::Steering
+          finish_interactive_markdown_deltas(markdown_chunks, stream_state)
+          print_user_transcript(event.input)
         when Events::SteeringApplied
           @prompt.clear_steered_count if @prompt.respond_to?(:clear_steered_count)
         when Events::Retry
@@ -104,7 +107,6 @@ module Kward
         when Events::ToolCall
           stream_state[:streamed] = true
           finish_interactive_markdown_deltas(markdown_chunks, stream_state)
-          print_tool_call(event.tool_call)
         when Events::ToolResult
           stream_state[:streamed] = true
           finish_interactive_markdown_deltas(markdown_chunks, stream_state)

data/lib/kward/cli/memory_commands.rb

@@ -18,53 +18,53 @@ module Kward
         when "enable"
           manager.enable
           agent.conversation.refresh_system_message!
-          @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Memory enabled.\n")
+          runtime_output("Memory enabled.")
         when "disable"
           manager.disable
           agent.conversation.memory_context = nil
           agent.conversation.refresh_system_message!
-          @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Memory disabled.\n")
+          runtime_output("Memory disabled.")
         when "auto-summary"
           case rest.to_s.strip
           when "enable", "on"
             manager.auto_summary_enable
-            @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Memory auto-summary enabled.\n")
+            runtime_output("Memory auto-summary enabled.")
           when "disable", "off"
             manager.auto_summary_disable
-            @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Memory auto-summary disabled.\n")
+            runtime_output("Memory auto-summary disabled.")
           else
-            @prompt.say("\nUsage: /memory auto-summary enable|disable\n")
+            runtime_output("Usage: /memory auto-summary enable|disable")
           end
         when "core"
           record = manager.add_core(unquote_argument(rest))
-          @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Added core memory #{record["id"]}.\n")
+          runtime_output("Added core memory #{record["id"]}.")
         when "add"
           record = manager.add_soft(unquote_argument(rest), scope: "workspace:#{agent.conversation.workspace_root}")
-          @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Added soft memory #{record["id"]}.\n")
+          runtime_output("Added soft memory #{record["id"]}.")
         when "list"
-          @prompt.say("\n#{format_memory_list(manager.hierarchy(workspace_root: agent.conversation.workspace_root))}\n")
+          runtime_output(format_memory_list(manager.hierarchy(workspace_root: agent.conversation.workspace_root)))
         when "forget"
           forgotten = manager.forget_memory(rest.to_s.strip)
-          @prompt.say("\n#{forgotten ? "Forgot #{rest.to_s.strip}." : "No memory found for #{rest.to_s.strip}."}\n")
+          runtime_output(forgotten ? "Forgot #{rest.to_s.strip}." : "No memory found for #{rest.to_s.strip}.")
         when "promote"
           record = manager.promote_memory(rest.to_s.strip)
-          @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Promoted memory #{record["id"]}.\n")
+          runtime_output("Promoted memory #{record["id"]}.")
         when "relax"
           record = manager.relax_core(rest.to_s.strip, workspace_root: agent.conversation.workspace_root)
-          @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Relaxed memory #{record["id"]}.\n")
+          runtime_output("Relaxed memory #{record["id"]}.")
         when "inspect"
-          @prompt.say("\n#{JSON.pretty_generate(manager.inspect_memory)}\n")
+          runtime_output(JSON.pretty_generate(manager.inspect_memory))
         when "why"
           explanation = agent.conversation.last_memory_retrieval || manager.explain_retrieval
-          @prompt.say("\n#{format_memory_why(explanation)}\n")
+          runtime_output(format_memory_why(explanation))
         when "summarize", "learn"
           records = summarize_memory(agent.conversation, manager: manager)
-          @prompt.say("\n#{colored(assistant_output_prompt, :green, :bold)} Learned #{records.length} soft #{records.length == 1 ? "memory" : "memories"}.\n")
+          runtime_output("Learned #{records.length} soft #{records.length == 1 ? "memory" : "memories"}.")
         else
-          @prompt.say("\nUsage: /memory enable|disable|auto-summary enable|disable|core <text>|add <text>|list|forget <id>|promote <id>|relax <id>|inspect|why|summarize\n")
+          runtime_output("Usage: /memory enable|disable|auto-summary enable|disable|core <text>|add <text>|list|forget <id>|promote <id>|relax <id>|inspect|why|summarize")
         end
       rescue StandardError => e
-        @prompt.say("\nMemory command failed: #{e.message}\n")
+        runtime_output("Memory command failed: #{e.message}")
       end
 
       def summarize_memory(conversation, manager: Memory::Manager.new)

data/lib/kward/cli/plugins.rb

@@ -26,7 +26,7 @@ module Kward
         @plugin_registry = PluginRegistry.load(reserved_commands: reserved_slash_command_names)
         conversation.plugin_registry = @plugin_registry if conversation.respond_to?(:plugin_registry=)
         conversation.refresh_system_message! if conversation.respond_to?(:refresh_system_message!)
-        @prompt.say("\nPlugins reloaded.\n")
+        runtime_output("Plugins reloaded.")
       end
 
       def reserved_slash_command_names
@@ -62,7 +62,7 @@ module Kward
         command.handler.call(argument, context)
         [true, nil]
       rescue StandardError => e
-        @prompt.say("\nPlugin command /#{name} error: #{e.message}\n")
+        runtime_output("Plugin command /#{name} error: #{e.message}")
         [true, nil]
       end
 
@@ -72,7 +72,7 @@ module Kward
           args: args,
           session: @active_session,
           workspace_root: conversation.workspace_root,
-          say_callback: lambda { |message| @prompt.say("\n#{message}\n") }
+          say_callback: lambda { |message| runtime_output(message) }
         )
       end
 

data/lib/kward/cli/prompt_interface.rb

@@ -52,10 +52,12 @@ module Kward
       end
 
       def prompt_footer_renderer
-        renderer = plugin_registry.footer_renderer
-        return nil unless renderer
+        return nil unless plugin_registry.footer_renderer
 
         lambda do
+          renderer = plugin_registry.footer_renderer
+          next "" unless renderer
+
           context = plugin_context(current_footer_conversation, "")
           renderer.call(context).to_s
         rescue StandardError => e
@@ -65,9 +67,10 @@ module Kward
       end
 
       def composer_status_text
-        provider = @client.respond_to?(:current_provider) ? @client.current_provider : "Codex"
-        model = @client.respond_to?(:current_model) ? @client.current_model : ModelInfo::DEFAULT_OPENAI_MODEL
-        reasoning = @client.respond_to?(:current_reasoning_effort) ? @client.current_reasoning_effort : ModelInfo::DEFAULT_REASONING_EFFORT
+        conversation = current_footer_conversation
+        provider = conversation.provider || (@client.respond_to?(:current_provider) ? @client.current_provider : "Codex")
+        model = conversation.model || (@client.respond_to?(:current_model) ? @client.current_model : ModelInfo::DEFAULT_OPENAI_MODEL)
+        reasoning = conversation.reasoning_effort || (@client.respond_to?(:current_reasoning_effort) ? @client.current_reasoning_effort : ModelInfo::DEFAULT_REASONING_EFFORT)
         reasoning = "n/a" unless ModelInfo.reasoning_supported?(provider, model) && !reasoning.to_s.empty?
         text = "#{provider} #{model} · #{reasoning}"
         parts = []
@@ -97,19 +100,18 @@ module Kward
         ANSI.colorize("#{value}%", color, enabled: @color_enabled)
       end
 
-      def composer_context_window
-        provider = @client.respond_to?(:current_provider) ? @client.current_provider : "Codex"
-        model = @client.respond_to?(:current_model) ? @client.current_model : ModelInfo::DEFAULT_OPENAI_MODEL
-        provider = ModelInfo.provider_label(provider)
-        @client.respond_to?(:current_context_window) ? @client.current_context_window : ModelInfo.context_window(provider, model)
+      def composer_context_window(provider = nil, model = nil)
+        provider ||= current_footer_conversation.provider || (@client.respond_to?(:current_provider) ? @client.current_provider : "Codex")
+        model ||= current_footer_conversation.model || (@client.respond_to?(:current_model) ? @client.current_model : ModelInfo::DEFAULT_OPENAI_MODEL)
+        ModelInfo.context_window(ModelInfo.provider_label(provider), model)
       end
 
       def composer_context_usage(provider, model)
-        context_window = composer_context_window
+        context_window = composer_context_window(provider, model)
         context_parts = if @client.respond_to?(:current_context_parts)
-                          @client.current_context_parts(current_footer_conversation.messages, footer_tool_schemas)
+                          @client.current_context_parts(current_footer_conversation.context_messages, footer_tool_schemas)
                         else
-                          { provider: provider, model: model, messages: current_footer_conversation.messages, tools: footer_tool_schemas }
+                          { provider: provider, model: model, messages: current_footer_conversation.context_messages, tools: footer_tool_schemas }
                         end
         @context_usage.call(
           provider: provider,

data/lib/kward/cli/rendering.rb

@@ -8,7 +8,7 @@ module Kward
 
       def render_conversation_transcript(conversation)
         tool_calls_by_id = {}
-        @prompt.say("\n#{colored("Transcript", :cyan, :bold)}\n")
+        @prompt.say("\n#{colored("Transcript", :gray, :bold)}\n")
         conversation.messages.each do |message|
           role = message_role(message)
           next if role == "system"
@@ -26,7 +26,6 @@ module Kward
             render_assistant_message(message)
             message_tool_calls(message).each do |tool_call|
               tool_calls_by_id[tool_call_id(tool_call)] = tool_call
-              render_tool_call(tool_call)
             end
           when "tool"
             render_tool_message(message, tool_calls_by_id)
@@ -44,7 +43,7 @@ module Kward
       end
 
       def render_assistant_message(message)
-        content = CLITranscriptFormatter.content_text(message_content(message))
+        content = CLITranscriptFormatter.assistant_content_text(message)
         return if content.empty?
 
         render_transcript_block("Assistant", content)
@@ -55,20 +54,12 @@ module Kward
         render_tool_result(tool_call, message_content(message).to_s)
       end
 
-      def render_tool_call(tool_call)
-        if prompt_interface?
-          print_tool_call(tool_call)
-        else
-          @prompt.say("\n#{colored("Tool>", :magenta, :bold)}\n#{tool_command(tool_call)}\n")
-        end
-      end
-
       def render_tool_result(tool_call, content)
         summary = limit_tool_output_lines(tool_result_summary(tool_call, content), INTERACTIVE_TOOL_OUTPUT_LINE_LIMIT)
         if prompt_interface?
           print_tool_result(tool_call, content, line_limit: INTERACTIVE_TOOL_OUTPUT_LINE_LIMIT)
         else
-          @prompt.say("\n#{colored("Tool output>", :cyan, :bold)}\n#{summary}\n")
+          @prompt.say("\n#{colored("Tool>", *tool_label_styles(content))}\n#{summary}\n")
         end
       end
 
@@ -80,7 +71,7 @@ module Kward
           print_block_delta(label, rendered)
           finish_stream_block
         else
-          @prompt.say("\n#{colored("#{transcript_label(label)}>", label_color(label), :bold)}\n#{rendered}\n")
+          @prompt.say("\n#{colored("#{transcript_label(label)}>", *label_styles(label))}\n#{rendered}\n")
         end
       end
 
@@ -102,7 +93,6 @@ module Kward
           :streamed
         when Events::ToolCall
           flush_markdown_deltas(markdown_chunks)
-          print_tool_call(event.tool_call)
           :streamed
         when Events::ToolResult
           flush_markdown_deltas(markdown_chunks)
@@ -304,24 +294,6 @@ module Kward
         RetryMessage.format(event)
       end
 
-      # Writes the tool call output for the terminal CLI flow.
-      def print_tool_call(tool_call)
-        if prompt_interface?
-          if @prompt.respond_to?(:write_stream_block)
-            @prompt.write_stream_block("Tool", "#{tool_command(tool_call)}\n", finish: true)
-          else
-            @prompt.start_stream_block("Tool")
-            @prompt.write_delta("#{tool_command(tool_call)}\n")
-            @prompt.finish_stream_block
-          end
-        else
-          start_stream_block("Tool")
-          puts tool_command(tool_call)
-          $stdout.flush
-          @stream_block = nil
-        end
-      end
-
       # Writes the tool result output for the terminal CLI flow.
       def print_tool_result(tool_call, content, line_limit: nil)
         summary = tool_result_summary(tool_call, content)
@@ -329,14 +301,14 @@ module Kward
         if prompt_interface?
           summary = summary.end_with?("\n") ? summary : "#{summary}\n"
           if @prompt.respond_to?(:write_stream_block)
-            @prompt.write_stream_block("Tool output", summary, finish: true)
+            @prompt.write_stream_block("Tool", summary, finish: true)
           else
-            @prompt.start_stream_block("Tool output")
+            @prompt.start_stream_block("Tool")
             @prompt.write_delta(summary)
             @prompt.finish_stream_block
           end
         else
-          start_stream_block("Tool output")
+          start_stream_block(tool_stream_label(content))
           print summary
           puts unless summary.end_with?("\n")
           $stdout.flush
@@ -348,7 +320,7 @@ module Kward
         return if @stream_block == label
 
         puts if @stream_block
-        puts "\n#{colored("#{transcript_label(label)}>", label_color(label), :bold)}"
+        puts "\n#{colored("#{transcript_label(label)}>", *label_styles(label))}"
         @stream_block = label
       end
 
@@ -366,24 +338,41 @@ module Kward
       end
 
       def transcript_label(label)
-        label == "Assistant" ? assistant_prompt_name : label
+        case label
+        when "Assistant"
+          assistant_prompt_name
+        when "Tool failed"
+          "Tool"
+        else
+          label
+        end
       end
 
-      def label_color(label)
+      def label_styles(label)
         case label
-        when "Reasoning"
-          :yellow
+        when "Reasoning", "Compaction summary"
+          [:gray, :bold]
         when "Assistant", "Kward"
-          :green
-        when "Tool"
-          :magenta
-        when "Tool output"
-          :cyan
+          [:green, :bold]
+        when "Tool", "Tool output"
+          [:cyan, :bold]
+        when "Tool failed"
+          [:red, :bold]
+        when "Retry"
+          [:yellow, :bold]
         else
-          :blue
+          [:gray, :bold]
         end
       end
 
+      def tool_stream_label(content)
+        tool_result_failed?(content) ? "Tool failed" : "Tool"
+      end
+
+      def tool_label_styles(content)
+        label_styles(tool_stream_label(content))
+      end
+
     end
   end
 end

data/lib/kward/cli/runtime_helpers.rb

@@ -31,6 +31,17 @@ module Kward
         @assistant_prompt || "Assistant>"
       end
 
+      def runtime_output_prompt
+        "Runtime>"
+      end
+
+      def runtime_output(text)
+        content = text.to_s.chomp
+        label = colored(runtime_output_prompt, :gray, :bold)
+        separator = content.include?("\n") ? "\n" : " "
+        @prompt.say("\n#{label}#{separator}#{content}\n")
+      end
+
       def build_interactive_agent(conversation)
         conversation.plugin_registry ||= plugin_registry if conversation.respond_to?(:plugin_registry)
         workspace = configured_workspace(root: conversation.workspace_root)
@@ -47,13 +58,13 @@ module Kward
       def handle_interactive_shell_command(input, agent)
         command = input.to_s.sub(/\A!\s*/, "")
         if command.strip.empty?
-          @prompt.say("\nShell command is required after !\n")
+          runtime_output("Shell command is required after !")
           return true
         end
 
         run_busy_local_command_and_requeue(activity: "running") do
           result = configured_workspace(root: interactive_workspace_root(agent)).run_shell_command(command)
-          @prompt.say("\n#{colored("Shell>", :green, :bold)} #{command}\n#{result}\n")
+          @prompt.say("\n#{colored("Shell>", :cyan, :bold)} #{command}\n#{result}\n")
         end
         true
       end