openclacky 1.2.12 → 1.2.14

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 451817565cffdf7b1efcdf5e741cea76af0451a8d9900804e2aa3c6a5384ba4a
-  data.tar.gz: 0232ede01332162004abc1638a8a03b41095c44c68198ce6327e5f5fc815f49a
+  metadata.gz: 82874a3ac7c623672bd09b5fa1be1c5dd70b1f223119a1b58b86f85417e46f1c
+  data.tar.gz: ba5f1cc02f50a0bee31e24a6ad009c265881eef8b8b9efa6f17b5bec29124414
 SHA512:
-  metadata.gz: 3a88a963b238a35fc25d5791b752981179290a88b27d176285647668711b91360a1d4c656677536fda84d18d0fa05fe67ad8364bdf0f7dbbba0a31a007156cbd
-  data.tar.gz: 124b77cbeec34494c8d35d58f1735459ba50da7c112a13a35c8038bbe89ee81412cf60107f4f926ad870efbbf65621b369f4bb5f1de67b477032b14dbad338d3
+  metadata.gz: 5535350a83909fffe2471ab0f6505d54f9bc2436826636eacb1c8d6bbbd84e554087b31b9503d6671449789160072eb743711556f302dc42b820637b7edab83d
+  data.tar.gz: bcdec5ed7e56cfc27ee2370ae46582239fa425254e013a7577f162b28c6e2d88b821768f2e367fe2b33dd6773804740692f1f29cba0ca2d73f40d38f6b8e2243

data/.clacky/skills/gem-release/SKILL.md

@@ -25,7 +25,7 @@ Automates the complete openclacky gem release workflow via `SKILL_DIR/scripts/re
 The release script (`SKILL_DIR/scripts/release.sh`) handles everything end-to-end:
 
 1. Pre-release checks (clean working directory, required tools)
-2. Run test suite (`bundle exec rspec`)
+2. Run test suite (`bundle exec rspec`) + web search smoke tests (real network — verifies Bing/DDG parsers still work against live HTML)
 3. Bump version in `lib/clacky/version.rb`
 4. Update `Gemfile.lock` via `bundle install`
 5. Commit and push to origin, wait for CI
@@ -177,6 +177,10 @@ Ask the user whether to use `--update-latest` before running the script.
 The script uses `set -euo pipefail` and stops on any failure. Common issues:
 
 - **Tests fail** → fix tests before re-running
+- **Web search smoke test fails (Bing)** → This often happens due to datacenter IP fingerprinting (anti-scrape blocking) returning irrelevant top-domain filler (like Mr.Bricolage). If you see "No ruby-related result from bing" during the smoke test:
+  1. Manually run `bundle exec rspec spec/integration/web_search_smoke_spec.rb --tag smoke` to verify
+  2. If it's the anti-scrape block, temporarily edit `spec/integration/web_search_smoke_spec.rb` to skip the relevance check on failure (e.g., using `skip "Bing returned anti-scrape garbage..."`)
+  3. Commit the change ("ci: skip bing smoke test relevance check on anti-scrape") and re-run the release script
 - **CI fails** → script pushes then watches CI; fix and re-push if needed
 - **gem push fails** → check RubyGems credentials (`gem signin`)
 - **gh release fails** → check `gh auth status`

data/.clacky/skills/gem-release/scripts/release.sh

@@ -116,10 +116,13 @@ step 2 "Running test suite"
 
 if [[ "$DRY_RUN" == true ]]; then
     echo -e "  ${YELLOW}[dry-run]${NC} bundle exec rspec"
+    echo -e "  ${YELLOW}[dry-run]${NC} bundle exec rspec spec/integration/web_search_smoke_spec.rb --tag smoke"
 else
     bundle exec rspec || die "Tests failed — aborting release"
+    bundle exec rspec spec/integration/web_search_smoke_spec.rb --tag smoke \
+        || die "Web search smoke tests failed — a provider parser may be broken on real network. Aborting release."
 fi
-success "All tests passed"
+success "All tests passed (including web search smoke)"
 
 # ════════════════════════════════════════════════════════════════════════
 # Step 3: Bump version

data/CHANGELOG.md

@@ -5,6 +5,45 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.2.14] - 2026-06-08
+
+### Added
+- OCR support for scanned PDFs (optical character recognition)
+- VLM-based PDF parser for improved document understanding
+
+### Improved
+- PDF OCR processing quality
+
+### Fixed
+- PDF processing not appearing in session history
+- Stale progress indicator that wouldn't dismiss
+
+### More
+- Document Bing smoke test anti-scrape failure handling in gem-release
+
+## [1.2.13] - 2026-06-08
+
+### Added
+- Session forking capability (Fork any message to a new session)
+- Gemini Flash 3.5 support and MIMO model pricing
+- Web search content capability and search skill LRU caching
+- Token usage visibility after tool calls
+- Subagent UI formatting for better readability
+
+### Improved
+- Web search performance using Bing race search strategy
+- Input box automatically clears when switching sessions
+- Skill evolution info display simplified
+- TUI adds an extra progress bar for better visual feedback
+
+### Fixed
+- Dir-picker path input synchronization on directory navigation
+- Thinking mode silent retries
+- IME (Input Method Editor) input check issues
+- WebUI reflect bug
+- Upstream JSON loading stability
+- Prevent skill evolution when the last message is incomplete
+
 ## [1.2.12] - 2026-06-05
 
 ### Fixed

data/lib/clacky/agent/llm_caller.rb

@@ -144,6 +144,28 @@ module Clacky
             raise RetryableError, "[LLM] Model returned empty response (no content, no tool_calls), retrying..."
           end
 
+          # Thinking-mode silent response detector. DeepSeek V4 / Kimi K2 /
+          # other reasoning models occasionally spend all output tokens inside
+          # `reasoning_content` and emit `content=""` + no tool_calls +
+          # `finish_reason="stop"`. Protocol-legal under OpenAI semantics
+          # (stop = model done), but semantically the model "thought and went
+          # silent" — agent main loop would treat it as task completion and
+          # exit. Reuse RetryableError so the existing retry + fallback
+          # pipeline handles it identically to 5xx/429.
+          if response[:content].to_s.strip.empty? &&
+              (response[:tool_calls].nil? || response[:tool_calls].empty?) &&
+              response[:reasoning_content].to_s.strip.length > 0 &&
+              response[:finish_reason].to_s == "stop"
+            reasoning_str = response[:reasoning_content].to_s
+            Clacky::Logger.warn("llm.thinking_mode_silent_response_detected",
+              model: api_call_model,
+              reasoning_len: reasoning_str.length,
+              reasoning_tail: reasoning_str[-200, 200] || reasoning_str,
+              completion_tokens: response.dig(:token_usage, :completion_tokens)
+            )
+            raise RetryableError, "[LLM] Thinking-mode model produced reasoning but empty content/tool_calls, retrying..."
+          end
+
         rescue Faraday::TimeoutError => e
           # Faraday::TimeoutError on our non-streaming POST almost always means
           # the *response* took longer than the 300s read-timeout to come back —
@@ -612,17 +634,10 @@ module Clacky
       # stream mid-tool_use (observed with Anthropic at ~127 s TTFT under
       # load), OpenRouter does NOT surface an error — it emits a valid
       # `tool_calls[]` whose `arguments` is empty, `"{}"`, or non-parseable
-      # JSON. Without this check the agent would either execute the tool with
-      # empty args or (worse) silently exit thinking the task finished.
-      #
-      # Rule is deliberately narrow: we only intercept the case where the
-      # model streamed literally nothing into the tool_call arguments —
-      # i.e. `nil`, empty string, or the placeholder `"{}"`. Partial/invalid
-      # JSON (e.g. `{"path": "/tmp/x"`) is left to the existing
-      # ArgumentsParser → BadArgumentsError path, because the model already
-      # committed to specific values and feeding the parse error back as a
-      # tool_result lets it self-correct in one round-trip (faster than a
-      # blind retry from scratch).
+      # JSON. Without this check the agent would either execute the tool
+      # with empty args, or write the broken arguments string back into
+      # history and have the NEXT request rejected by the upstream proxy
+      # with a 400 BadRequest at the json.loads boundary.
       private def detect_upstream_truncation!(response)
         tool_calls = response[:tool_calls]
         return if tool_calls.nil? || tool_calls.empty?
@@ -653,22 +668,23 @@ module Clacky
           "(args=#{args_str[0, 40].inspect}). Retrying..."
       end
 
-      # True when a tool_call's arguments field looks COMPLETELY empty —
-      # i.e. the upstream stream was cut before the model wrote any real
-      # content into the arguments JSON.
+      # True when a tool_call's arguments field is unusable — either empty
+      # or not a complete, parseable JSON object.
       #
       # Rules:
-      #   - nil / non-String / empty string  → truncated (nothing at all)
+      #   - nil / non-String / empty string  → truncated
       #   - parses to {} (empty object)      → truncated (placeholder only)
-      #   - anything else (including partial/invalid JSON like `{"path":
-      #     "/tmp/x"` where the model already started writing) → NOT
-      #     truncated by this detector
+      #   - JSON::ParserError (partial JSON) → truncated
+      #   - valid non-empty JSON object      → NOT truncated
       #
-      # Partial-JSON cases are deliberately left to the existing
-      # ArgumentsParser → BadArgumentsError path, which surfaces the parse
-      # error back to the LLM as a tool_result so it can self-correct. That
-      # is more efficient than a blind retry when the model already wrote
-      # most of the args.
+      # Why partial JSON counts as truncated: even though ArgumentsParser
+      # could repair it for the current turn, the original broken string
+      # still ends up in history (agent.rb#format_tool_calls_for_api keeps
+      # arguments verbatim). The next turn's request body would then carry
+      # an invalid JSON in tool_calls[].function.arguments, which upstream
+      # proxies (LiteLLM, OpenRouter, etc.) reject with a 400 BadRequest
+      # before the model ever sees it. Retrying from a clean state is the
+      # only path that actually recovers.
       private def tool_call_args_truncated?(args)
         return true if args.nil?
         return true unless args.is_a?(String)
@@ -677,8 +693,7 @@ module Clacky
         parsed = begin
           JSON.parse(args)
         rescue JSON::ParserError
-          # Partial/invalid JSON — let ArgumentsParser handle it downstream.
-          return false
+          return true
         end
 
         parsed.is_a?(Hash) && parsed.empty?

data/lib/clacky/agent/memory_updater.rb

@@ -68,6 +68,18 @@ module Clacky
       def run_memory_update_subagent
         return unless should_update_memory?
 
+        with_memory_update_phase do
+          run_memory_update_subagent_inner
+        end
+      end
+
+      private def with_memory_update_phase
+        return yield unless @ui.respond_to?(:with_phase)
+
+        @ui.with_phase(kind: "memory_update", label: "Updating long-term memory") { yield }
+      end
+
+      private def run_memory_update_subagent_inner
         handle = @ui&.start_progress(message: "Updating long-term memory…", style: :primary)
 
         # Fork subagent inheriting main agent's model, tools, and history.

data/lib/clacky/agent/session_serializer.rb

@@ -272,6 +272,7 @@ module Clacky
           # Disk files (PDF, doc, etc.): stored in display_files on the user message at send time
           disk_files  = Array(msg[:display_files]).map { |f|
             { name: f[:name] || f["name"], type: f[:type] || f["type"] || "file",
+              path: f[:path] || f["path"],
               preview_path: f[:preview_path] || f["preview_path"] }
           }
           all_files = image_files + disk_files

data/lib/clacky/agent/skill_auto_creator.rb

@@ -73,11 +73,14 @@ module Clacky
 
           ## Decision Criteria (ALL must be true)
 
-          1. **Reusable**: The workflow could apply to similar tasks in the future
+          1. **Turn is actually finished**: The assistant's last message is
+             not a question back to the user, and the user wasn't just asking
+             /discussing/exploring (Q&A is not work to capture).
+          2. **Reusable**: The workflow could apply to similar tasks in the future
              (not a one-off, project-specific task)
-          2. **Well-defined**: Clear steps with consistent logic, not just exploratory conversation
-          3. **Valuable**: Would save more than 5 minutes of work if reused
-          4. **Generalizable**: Can be parameterized for different inputs/contexts
+          3. **Well-defined**: Clear steps with consistent logic, not just exploratory conversation
+          4. **Valuable**: Would save more than 5 minutes of work if reused
+          5. **Generalizable**: Can be parameterized for different inputs/contexts
 
           ## Action
 

data/lib/clacky/agent/skill_evolution.rb

@@ -26,17 +26,35 @@ module Clacky
       def run_skill_evolution_hooks
         return unless skill_evolution_enabled?
         return if @is_subagent
+        return unless skill_evolution_visible? || skill_evolution_has_work?
 
+        with_skill_evolution_phase do
+          if @skill_execution_context
+            maybe_reflect_on_skill
+          else
+            maybe_create_skill_from_task
+          end
+        end
+      end
+
+      private def skill_evolution_visible?
+        @config.respond_to?(:verbose) && @config.verbose
+      end
+
+      private def skill_evolution_has_work?
         if @skill_execution_context
-          # Scenario 2: Reflect on executed skill (may invoke skill-creator
-          # to UPDATE the existing skill, but will not create a new one).
-          maybe_reflect_on_skill
+          should_reflect_on_skill?
         else
-          # Scenario 1: Auto-create new skill from complex task.
-          maybe_create_skill_from_task
+          should_auto_create_skill?
         end
       end
 
+      private def with_skill_evolution_phase
+        return yield unless @ui.respond_to?(:with_phase)
+
+        @ui.with_phase(kind: "skill_evolution", label: "Reflecting on this task") { yield }
+      end
+
       # Check if skill evolution is enabled in config
       # @return [Boolean]
       private def skill_evolution_enabled?

data/lib/clacky/agent/skill_manager.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require "fileutils"
+
 module Clacky
   class Agent
     # Skill management and execution
@@ -128,6 +130,32 @@ module Clacky
           s.identifier.to_s.start_with?("mcp:")
         end
 
+        # Sort normal skills so AVAILABLE SKILLS prioritises what the user
+        # actually relies on:
+        #   1. default skills first (alphabetical, stable) — the always-present
+        #      built-in baseline; they don't participate in LRU.
+        #   2. user-installed (project + brand + global) after, ordered by the
+        #      skill directory's mtime descending (LRU). touch_skill_for_lru
+        #      bumps mtime on every invocation; freshly installed skills also
+        #      naturally float to the top.
+        #   3. search-skills is pinned to the very end (after truncation) so it
+        #      sits next to the "(N more skills installed)" hint and is the
+        #      last thing the LLM sees when scanning the list — maximising the
+        #      chance it remembers to search before building a duplicate skill.
+        default_skills, user_skills = normal_skills.partition { |s| s.source == :default }
+        search_skill, default_skills = default_skills.partition { |s| s.identifier.to_s == "search-skills" }
+        default_skills = default_skills.sort_by { |s| s.identifier.to_s }
+        user_skills = user_skills.sort_by { |s|
+          mt = File.mtime(s.directory.to_s).to_f rescue 0.0
+          [-mt, s.identifier.to_s]
+        }
+        normal_skills = default_skills + user_skills
+
+        # Track total before truncation so we can hint the agent that more
+        # skills exist beyond the window.
+        total_normal_skills = normal_skills.size
+        truncated_skill_count = 0
+
         # Enforce system prompt injection limit to control token usage.
         # Warn at most once per process per dropped-set signature — build_skill_context
         # runs on every system-prompt assembly and is invoked from many short-lived
@@ -135,6 +163,7 @@ module Clacky
         if normal_skills.size > MAX_CONTEXT_SKILLS
           kept    = normal_skills.first(MAX_CONTEXT_SKILLS)
           dropped = normal_skills.drop(MAX_CONTEXT_SKILLS)
+          truncated_skill_count = dropped.size
           dropped_names = dropped.map(&:identifier)
           signature = dropped_names.sort.join(",")
 
@@ -150,6 +179,8 @@ module Clacky
           normal_skills = kept
         end
 
+        normal_skills += search_skill unless search_skill.empty?
+
         if mcp_skills.size > MAX_CONTEXT_MCP_SERVERS
           dropped = mcp_skills.drop(MAX_CONTEXT_MCP_SERVERS).map(&:identifier)
           signature = "mcp:" + dropped.sort.join(",")
@@ -194,6 +225,12 @@ module Clacky
             end
           end
 
+          if truncated_skill_count > 0
+            context += "(#{truncated_skill_count} more skill(s) installed but not shown here. " \
+                       "If the listed skills don't fit the task, invoke the `search-skills` skill " \
+                       "to look them up by keyword BEFORE deciding to build a new skill.)\n\n"
+          end
+
           context += "\n"
           sections << context
         end
@@ -296,6 +333,8 @@ module Clacky
       # @param task_id [Integer] Current task ID (for message tagging)
       # @return [void]
       def inject_skill_as_assistant_message(skill, arguments, task_id, slash_command: false)
+        touch_skill_for_lru(skill)
+
         # Track skill execution context for self-evolution system
         @skill_execution_context = {
           skill_name: skill.identifier,
@@ -413,10 +452,42 @@ module Clacky
       # @return [Hash<String, Proc>]
       def build_template_context
         {
-          "memories_meta" => -> { load_memories_meta }
+          "memories_meta"   => -> { load_memories_meta },
+          "all_skills_meta" => -> { load_all_skills_meta }
         }
       end
 
+      # Render a complete list of installed skills (no MAX_CONTEXT_SKILLS cap)
+      # for skills like `search-skills` that need to see every available skill.
+      # Brand skill names + descriptions are pulled from cached_metadata so this
+      # is safe to inject without touching encrypted SKILL.md.enc content.
+      # @return [String]
+      def load_all_skills_meta
+        all = @skill_loader.load_all
+        all = filter_skills_by_profile(all)
+        all = all.reject(&:invalid?)
+        all = all.reject { |s| s.identifier.to_s.start_with?("mcp:") }
+
+        return "(No skills installed.)" if all.empty?
+
+        default_skills, user_skills = all.partition { |s| s.source == :default }
+        default_skills = default_skills.sort_by { |s| s.identifier.to_s }
+        user_skills = user_skills.sort_by { |s|
+          mt = File.mtime(s.directory.to_s).to_f rescue 0.0
+          [-mt, s.identifier.to_s]
+        }
+        ordered = default_skills + user_skills
+
+        lines = ["All installed skills (#{ordered.size} total):", ""]
+        ordered.each do |skill|
+          lines << "- name: #{skill.identifier}"
+          lines << "  source: #{skill.source}"
+          lines << "  description: #{skill.context_description}"
+          lines << ""
+        end
+        lines.join("\n")
+      end
+
       # Scan ~/.clacky/memories/ and return a formatted summary of all memory files.
       # Parses YAML frontmatter (same pattern as Skill#parse_frontmatter) for each file.
       # @return [String] Formatted list of memory topics and descriptions
@@ -488,11 +559,25 @@ module Clacky
         FileUtils.remove_dir(dir, true) rescue nil
       end
 
+      # Bump a skill's directory mtime so user-installed skills sort by recent
+      # use (LRU) when assembling AVAILABLE SKILLS. Touches the directory, NOT
+      # SKILL.md — the WebUI creator center uses SKILL.md mtime to detect local
+      # edits, and we must not produce false positives there.
+      # default-source skills are skipped: they don't participate in LRU and
+      # often live in a read-only gem path.
+      def touch_skill_for_lru(skill)
+        return if skill.source == :default
+        FileUtils.touch(skill.directory.to_s)
+      rescue StandardError
+        nil
+      end
+
       # Execute a skill in a forked subagent
       # @param skill [Skill] The skill to execute
       # @param arguments [String] Arguments for the skill
       # @return [String] Summary of subagent execution
       def execute_skill_with_subagent(skill, arguments)
+        touch_skill_for_lru(skill)
         # For encrypted brand skills with supporting scripts: decrypt to a tmpdir.
         # Subagent path has a clear boundary (subagent.run returns), so we shred inline
         # rather than registering on the parent agent.

data/lib/clacky/agent/skill_reflector.rb

@@ -19,45 +19,35 @@ module Clacky
       # Check if we should reflect on the skill that just executed
       # Called from SkillEvolution#run_skill_evolution_hooks
       def maybe_reflect_on_skill
-        return unless @skill_execution_context
-
-        # Only reflect on skills that the user explicitly invoked via slash command.
-        # Skills triggered by the LLM itself (e.g. as part of a broader task) or
-        # platform-management skills invoked incidentally should not be reflected on.
-        return unless @skill_execution_context[:slash_command]
-
-        # Skip default and brand skills — they are system-owned and should not be
-        # auto-improved by the evolution system.
-        source = @skill_execution_context[:source]
-        return if source == :default || source == :brand
+        return unless should_reflect_on_skill?
 
         skill_name = @skill_execution_context[:skill_name]
-        start_iteration = @skill_execution_context[:start_iteration]
-        
-        # Calculate iterations within the skill execution (not session-cumulative)
-        iterations = @iterations - start_iteration
-
-        # Only reflect if the skill actually ran for a meaningful number of iterations
-        return if iterations < MIN_SKILL_ITERATIONS
 
-        # Fork an isolated subagent to reflect + improve — does NOT touch main history
         @ui&.show_info("Reflecting on skill execution: #{skill_name}")
         subagent = fork_subagent
         result = subagent.run(build_skill_reflection_prompt(skill_name))
 
-        # Merge subagent cost into parent's cumulative session spend so the
-        # sessionbar reflects the real total. Without this, reflection cost
-        # silently disappears from the user's visible total.
         if result
           subagent_cost = result[:total_cost_usd] || 0.0
           @total_cost += subagent_cost
           @ui&.update_sessionbar(cost: @total_cost, cost_source: @cost_source)
         end
 
-        # Clear the context so we don't reflect again
         @skill_execution_context = nil
       end
 
+      private def should_reflect_on_skill?
+        return false unless @skill_execution_context
+        return false unless @skill_execution_context[:slash_command]
+
+        source = @skill_execution_context[:source]
+        return false if source == :default || source == :brand
+
+        start_iteration = @skill_execution_context[:start_iteration]
+        iterations = @iterations - start_iteration
+        iterations >= MIN_SKILL_ITERATIONS
+      end
+
       # Build the reflection prompt content
       # @param skill_name [String]
       # @return [String]
@@ -79,6 +69,11 @@ module Clacky
 
           ## Decision
 
+          If the assistant's last message is a question back to the user
+          (the turn isn't actually finished), or the user was just asking/
+          discussing rather than finishing a task:
+            → Respond briefly: "Skill #{skill_name} worked well, no improvements needed."
+
           If you identified **concrete, actionable improvements**:
             → Call invoke_skill("skill-creator", task: "Improve skill #{skill_name}: [describe specific improvements needed]")