openclacky 0.9.36 → 0.9.37

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8a7f74cba226bc2332676ed9f0a72c27f3fb6b45bafacd60198f7ec9b59a84c7
-  data.tar.gz: 7396d2258a38e203a5ffe217f0409fb101487bdb74dad8f224649a09b8adb775
+  metadata.gz: 951665db04cf6c2a4f8ef9b5c0555f4df91b84af08f63d21097d97cbd64d44b2
+  data.tar.gz: 82457a522007f54ecd5fcc36fee8e79cf16a65996dd9b95d7a00fd0dcfbc2cac
 SHA512:
-  metadata.gz: 7f10c234d4a22d825e3623bff7e0a0faf55af4cea34dc15a939fbde3e6e3792bf1a11cd97d68e0c6890dd37c011814019cfed7dea70c724fcb2fc11b49cff15a
-  data.tar.gz: 12dcd062214b42191f33e4b2e09ec62f24cb9f91206451ff23861672a2cc8b680d89e5498cf0da6c53b74d4da2933bd53863666cee608539edb398f016226df5
+  metadata.gz: 031b1031a702aca3a7cee36ea8ba23bc4982e95f8381e59d07ecb652432f6bc8a40e9a386e4a254d640f18f0088d9e18fa1f6434d92c038844f4821cef40a703
+  data.tar.gz: 5c5c36517efebb37a7a44d0531e0baadedb8600319682c39696a971771852019f2649386e15a3505d9ae32e86cf8c5818c64cf36943247220a7db6e0df24e994

data/CHANGELOG.md

@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.9.37] - 2026-04-24
+
+### Fixed
+- **Critical: pinned sessions could silently disappear from the sidebar** ("the pinned one isn't showing, and refreshing sometimes fixes it"). Root cause: the backend `list` endpoint only sorted by `created_at` and applied `limit` blindly, so a pinned session older than the first page's rows was cut off entirely — the frontend's `byPinnedAndTime` sort never saw it. "Refreshing sometimes worked" only if the pinned session happened to be recent enough to land in the first 20 rows. Fix: `SessionRegistry#list` now partitions results and **always returns ALL matching pinned sessions on the first page regardless of `limit`**, followed by up to `limit` non-pinned sessions. The `before` cursor applies only to the non-pinned section, so "load more" pages never re-send or duplicate pinned rows. `/api/sessions`'s `has_more` is now computed from non-pinned overflow only. Frontend `loadMore` cursor also excludes pinned rows so pagination jumps correctly. Regression specs cover: (a) an old pinned session still appears when `limit=3`, (b) multiple pinned sessions all fit on page one with `limit=1`, (c) pinned sessions never duplicate into `before`-cursor pages.
+- **Critical: saving one model in Web UI Settings silently wiped other models' API keys.** The 0.9.36 index→id refactor (commit `b61e22e`) rebuilt each model hash from scratch on save (`"api_key" => api_key.to_s`), dropping the old `existing["api_key"] = api_key if api_key` guard. Combined with `/api/config` returning only `api_key_masked` (never `api_key`), every non-edited row in the POST body arrived with `api_key: undefined` — the backend then rewrote those rows' keys to `""`. Now `api_save_config` has three explicit cases for resolving `api_key`: (1) masked placeholder → keep stored key, (2) **missing/blank on an existing row → keep stored key (this fix)**, (3) otherwise use incoming value. Brand-new models (no `id`) still create with an empty key as before.
+- **Critical: in-app upgrade no longer falsely reports failure.** The 0.9.36 upgrade flow shared a PTY helper (`run_shell`) with the new unified Terminal tool, which — by design — returns early with a `session_id` when command output stays quiet for 3 seconds. Long-running `gem install` operations routinely hit this during dependency resolution, causing the Web UI to show `✗ Upgrade failed.` even when the gem installed successfully. `run_shell` now delegates to a new `Terminal.run_sync` Ruby API that polls until the command truly completes, and `finish_upgrade` additionally re-checks the installed gem version as a defensive fallback.
+- **Critical: "历史记录获取失败 (500: source sequence is illegal/malformed utf-8)" when opening a session.** When `file_reader` / `edit` / `grep` / `glob` encountered a file with non-UTF-8 bytes (e.g. GBK-encoded text or a Chinese Windows-exported CSV), the dirty bytes flowed through tool results into the agent history and session chunks on disk. Later, when `GET /api/sessions/:id/messages` replayed that history, `JSON.generate` would blow up on the invalid byte sequence and return 500. Now every IO source point scrubs invalid bytes to U+FFFD (`�`) at read time: `file_reader` (both content and directory entry names), `edit`, `grep` (`File.foreach` + context `readlines`), `glob` (`Dir.glob` path strings), `session_serializer` (chunk md replay), and `tool_executor` (diff preview). A defense-in-depth layer in `MessageHistory#append` / `#replace_all` recursively sanitizes every string that enters the message tree — so even a future tool that forgets to scrub cannot poison the session.
+
+### Added
+- **New `Terminal.run_sync` internal API** for Ruby callers that need synchronous command capture (drop-in replacement for `Open3.capture2e`, but using the same PTY + login-shell + Security pipeline as the AI-facing tool).
+- **DeepSeek V4 provider preset.** New `deepseekv4` entry in `Clacky::Providers` (positioned right after `openrouter`) with default model `deepseek-v4-pro` and models list `deepseek-v4-flash`, `deepseek-v4-pro`, plus the deprecated-aliases `deepseek-chat` / `deepseek-reasoner` (to be removed on 2026-07-24). Uses the OpenAI-compatible endpoint `https://api.deepseek.com`; for Anthropic-format usage, point `base_url` at `https://api.deepseek.com/anthropic` and switch `api` to `anthropic-messages`.
+- **DeepSeek V4 pricing.** Added `deepseek-v4-flash` ($0.14 in / $0.28 out / $0.028 cache-hit per MTok) and `deepseek-v4-pro` ($1.74 in / $3.48 out / $0.145 cache-hit per MTok) to `Clacky::ModelPricing::PRICING_TABLE`. Legacy aliases `deepseek-chat` and `deepseek-reasoner` normalize to `deepseek-v4-flash`. DeepSeek has no separate cache-write charge, so cache writes are billed at the cache-miss (input) rate. Prices sourced from the official pricing page (USD per 1M tokens).
+
 ## [0.9.36] - 2026-04-24
 
 ### Fixed

data/lib/clacky/agent/session_serializer.rb

@@ -243,7 +243,12 @@ module Clacky
         end
         visited = visited.dup.add(canonical)
 
-        raw = File.read(resolved)
+        # Scrub invalid UTF-8 bytes defensively — chunk files written before
+        # the 0.9.37 fix may contain poisoned bytes from file_reader results.
+        raw = File.read(resolved).then do |s|
+          s.encoding == Encoding::UTF_8 && s.valid_encoding? ? s :
+            s.encode("UTF-8", invalid: :replace, undef: :replace, replace: "\u{FFFD}")
+        end
 
         # Parse YAML front matter to get archived_at for synthetic timestamps
         archived_at = nil

data/lib/clacky/agent/tool_executor.rb

@@ -358,6 +358,7 @@ module Clacky
           @ui&.show_diff("", new_content, max_lines: 50)
         else
           old_content = File.read(expanded_path)
+          old_content = old_content.encode("UTF-8", invalid: :replace, undef: :replace, replace: "\u{FFFD}") unless old_content.encoding == Encoding::UTF_8 && old_content.valid_encoding?
           @ui&.show_diff(old_content, new_content, max_lines: 50)
         end
         nil

data/lib/clacky/message_history.rb

@@ -34,7 +34,7 @@ module Clacky
       if message[:role] == "user"
         drop_dangling_tool_calls!
       end
-      @messages << message
+      @messages << deep_sanitize_utf8(message)
       self
     end
 
@@ -53,7 +53,7 @@ module Clacky
 
     # Replace the entire message list (used by compression rebuild).
     def replace_all(new_messages)
-      @messages = new_messages.dup
+      @messages = new_messages.map { |m| deep_sanitize_utf8(m) }
       self
     end
 
@@ -229,5 +229,46 @@ module Clacky
     private def strip_internal_fields(message)
       message.reject { |k, _| INTERNAL_FIELDS.include?(k) }
     end
+
+    # Defense-in-depth: recursively scrub invalid UTF-8 bytes from every String
+    # stored in the message tree. Even if a tool forgets to scrub its output,
+    # nothing poisoned will ever reach session persistence or JSON.generate.
+    #
+    # Fast path: if the tree contains only valid UTF-8 strings, the original
+    # object is returned unchanged — preserving object identity for callers
+    # that rely on `equal?` (e.g. rollback_before).
+    # Slow path: any invalid byte triggers a rebuild with scrubbed strings
+    # (invalid bytes → U+FFFD).
+    private def deep_sanitize_utf8(obj)
+      case obj
+      when String
+        return obj if obj.encoding == Encoding::UTF_8 && obj.valid_encoding?
+        obj.encode("UTF-8", invalid: :replace, undef: :replace, replace: "\u{FFFD}")
+      when Hash
+        return obj unless contains_dirty_utf8?(obj)
+        obj.transform_values { |v| deep_sanitize_utf8(v) }
+      when Array
+        return obj unless contains_dirty_utf8?(obj)
+        obj.map { |v| deep_sanitize_utf8(v) }
+      else
+        obj
+      end
+    end
+
+    # Cheap recursive check: does this subtree contain any invalid-UTF-8 string?
+    # Short-circuits on first offender. Keeps the common case (all valid UTF-8)
+    # allocation-free.
+    private def contains_dirty_utf8?(obj)
+      case obj
+      when String
+        !(obj.encoding == Encoding::UTF_8 && obj.valid_encoding?)
+      when Hash
+        obj.any? { |_, v| contains_dirty_utf8?(v) }
+      when Array
+        obj.any? { |v| contains_dirty_utf8?(v) }
+      else
+        false
+      end
+    end
   end
 end

data/lib/clacky/providers.rb

@@ -41,6 +41,27 @@ module Clacky
         "website_url" => "https://openrouter.ai/keys"
       }.freeze,
 
+      "deepseekv4" => {
+        "name" => "DeepSeek V4",
+        # DeepSeek API is compatible with both OpenAI and Anthropic formats.
+        # We use the OpenAI-compatible endpoint here (matches kimi/minimax/glm style).
+        # For Anthropic-format usage, point base_url at https://api.deepseek.com/anthropic
+        # and change "api" to "anthropic-messages".
+        "base_url" => "https://api.deepseek.com",
+        "api" => "openai-completions",
+        "default_model" => "deepseek-v4-pro",
+        # Note: deepseek-chat and deepseek-reasoner are legacy aliases being
+        # deprecated on 2026-07-24; they map to deepseek-v4-flash's non-thinking
+        # and thinking modes respectively. Prefer deepseek-v4-flash / deepseek-v4-pro.
+        "models" => [
+          "deepseek-v4-flash",
+          "deepseek-v4-pro",
+          "deepseek-chat",
+          "deepseek-reasoner"
+        ],
+        "website_url" => "https://platform.deepseek.com/api_keys"
+      }.freeze,
+
       "minimax" => {
         "name" => "Minimax",
         "base_url" => "https://api.minimaxi.com/v1",

data/lib/clacky/server/http_server.rb

@@ -468,10 +468,19 @@ module Clacky
         # Backward-compat: ?source=<x> and ?profile=coding → type
         type ||= query["profile"].to_s.strip.then { |v| v.empty? ? nil : v }
         type ||= query["source"].to_s.strip.then  { |v| v.empty? ? nil : v }
-        # Fetch one extra to detect has_more without a separate count query
+
+        # Fetch one extra NON-PINNED row to detect has_more without a separate count query.
+        # `registry.list` always returns ALL matching pinned rows first (on the
+        # first page; `before` == nil), followed by non-pinned rows up to `limit+1`.
+        # So has_more is determined by whether the non-pinned section overflowed.
         sessions = @registry.list(limit: limit + 1, before: before, q: q, date: date, type: type)
-        has_more = sessions.size > limit
-        sessions = sessions.first(limit)
+
+        # Split pinned vs non-pinned to apply has_more only to the non-pinned tail.
+        pinned_part, non_pinned_part = sessions.partition { |s| s[:pinned] }
+        has_more = non_pinned_part.size > limit
+        non_pinned_part = non_pinned_part.first(limit)
+        sessions = pinned_part + non_pinned_part
+
         json_response(res, 200, { sessions: sessions, has_more: has_more })
       end
 
@@ -930,8 +939,12 @@ module Clacky
         key.empty? ? nil : key
       end
 
-      # Extract bearer token / query param / cookie from a WEBrick request.
-      # Priority: Authorization: Bearer > ?access_key= > Cookie clacky_access_key
+      # Extract bearer token or query param from a WEBrick request.
+      # Priority: Authorization: Bearer > ?access_key=
+      # The query string form is only used by WebSocket connections, which
+      # cannot set custom headers from the browser. All HTTP clients —
+      # including the web UI (via a fetch interceptor in auth.js) — use the
+      # Authorization header.
       private def extract_key(req)
         auth = req["Authorization"].to_s.strip
         if auth.start_with?("Bearer ")
@@ -943,10 +956,6 @@ module Clacky
         token = query["access_key"].to_s.strip
         return token unless token.empty?
 
-        req.cookies.each do |c|
-          return c.value if c.name == "clacky_access_key" && !c.value.to_s.empty?
-        end
-
         nil
       end
 
@@ -1119,7 +1128,18 @@ module Clacky
       end
 
       # Broadcast final upgrade result with appropriate log message.
+      #
+      # Defensive post-check: if `run_shell` reported failure but the gem
+      # is in fact now installed at the latest version, reverse the verdict.
+      # This guards against false negatives from the Terminal idle-poll
+      # mechanism (see: 0.9.36 upgrade failure bug).
       private def finish_upgrade(success, fallback_hint: "gem update openclacky")
+        if !success && gem_actually_upgraded?
+          Clacky::Logger.warn("[Upgrade] run_shell reported failure, but installed version matches latest — treating as success.")
+          broadcast_all(type: "upgrade_log", line: "\n(Verified: the new version is installed — reclassifying as success.)\n")
+          success = true
+        end
+
         if success
           Clacky::Logger.info("[Upgrade] Success!")
           broadcast_all(type: "upgrade_log", line: "\n✓ Upgrade successful! Please restart the server to apply the new version.\n")
@@ -1131,6 +1151,22 @@ module Clacky
         end
       end
 
+      # Check whether the latest published version of openclacky is already
+      # installed locally. Used as a post-upgrade sanity check so a flaky
+      # run_shell result doesn't mask a successful install.
+      # Returns false on any error (conservative — don't fabricate success).
+      private def gem_actually_upgraded?
+        latest = fetch_latest_version_from_rubygems_api
+        return false unless latest
+
+        out, exit_code = run_shell("gem list openclacky -i -v #{latest}", timeout: 30)
+        return false unless exit_code&.zero?
+        out.to_s.strip.downcase == "true"
+      rescue StandardError => e
+        Clacky::Logger.warn("[Upgrade] gem_actually_upgraded? error: #{e.message}")
+        false
+      end
+
       # POST /api/restart
       # Re-execs the current process so the newly installed gem version is loaded.
       # Uses the absolute script path captured at startup to avoid relative-path issues.
@@ -1242,18 +1278,11 @@ module Clacky
       # Run a shell command via the unified Terminal tool and return
       # [output, exit_code] — drop-in replacement for Open3.capture2e.
       #
-      # Uses Terminal#execute so the command inherits the user's real
-      # login shell (rbenv/mise shims, configured gem mirrors, etc.).
-      # On timeout / still-running, returns [output_so_far, nil].
-      #
-      # The command is routed through the Security layer like any other
-      # Terminal call; server-side commands (`gem ...`, `curl -fsSL ... -o ...`)
-      # pass through unchanged.
+      # Delegates to Terminal.run_sync which handles the idle-poll loop
+      # internally (see its docs for why that's needed — this wrapper used
+      # to re-implement it wrong and caused the 0.9.36 upgrade bug).
       private def run_shell(command, timeout: 120)
-        result = Clacky::Tools::Terminal.new.execute(command: command, timeout: timeout)
-        output    = result[:output].to_s
-        exit_code = result[:exit_code] # nil when the session is still running
-        [output, exit_code]
+        Clacky::Tools::Terminal.run_sync(command, timeout: timeout)
       end
 
       # ── Channel API ───────────────────────────────────────────────────────────
@@ -1915,18 +1944,32 @@ module Clacky
           # Lookup by id only. No id means a brand-new model — we mint one.
           existing = m["id"] && existing_by_id[m["id"]]
 
-          # Resolve api_key: if masked placeholder, keep the stored key
-          api_key = if m["api_key"].to_s.include?("****")
-                      existing&.dig("api_key")
+          # Resolve api_key with THREE cases (ordered, fail-safe):
+          #   1. Incoming key is the masked placeholder ("sk-ab12****...5678")
+          #      → user didn't retype; keep the stored key.
+          #   2. Incoming key is missing/blank AND we have an existing model
+          #      → the browser omitted api_key for non-edited rows; keep the
+          #        stored key. This is the critical 0.9.37 fix — without it,
+          #        saving one model silently wiped api_keys of all others,
+          #        because the frontend only ever hydrates api_key for the
+          #        row currently being edited (/api/config returns only
+          #        api_key_masked, never api_key).
+          #   3. Otherwise: user typed a new key (or this is a brand-new
+          #      model); use the incoming value.
+          incoming_key = m["api_key"].to_s
+          api_key = if incoming_key.include?("****")
+                      existing&.dig("api_key").to_s
+                    elsif incoming_key.empty? && existing
+                      existing["api_key"].to_s
                     else
-                      m["api_key"]
+                      incoming_key
                     end
 
           {
             "id"               => (existing && existing["id"]) || SecureRandom.uuid,
             "model"            => m["model"].to_s,
             "base_url"         => m["base_url"].to_s,
-            "api_key"          => api_key.to_s,
+            "api_key"          => api_key,
             "anthropic_format" => m["anthropic_format"] || false,
             "type"             => m["type"]
           }.tap { |h| h.delete("type") if h["type"].nil? || h["type"].to_s.empty? }

data/lib/clacky/server/session_registry.rb

@@ -143,11 +143,22 @@ module Clacky
       #            nil = no source filter (all sessions)
       #   profile: "general"|"coding"|nil
       #            nil = no agent_profile filter
-      #   limit:   max sessions to return
+      #   limit:   max sessions to return (applies to NON-PINNED only; see below)
       #   before:  ISO8601 cursor — only sessions with created_at < before
+      #             (also applies to NON-PINNED only; pinned items are a separate
+      #             logical section, they should never be paginated away)
+      #   include_pinned: when true (default), all matching pinned sessions are
+      #             always returned on the FIRST page (before == nil) regardless
+      #             of limit. Subsequent pages (before set) contain only
+      #             non-pinned sessions. This guarantees that users who pinned
+      #             an old session always see it at the top of the sidebar,
+      #             even if many newer sessions exist.
+      #
+      # Ordering of the returned array:
+      #   [ ...all_pinned_matching (newest-first), ...non_pinned (newest-first, limited) ]
       #
       # source and profile are orthogonal — either can be nil independently.
-      def list(limit: nil, before: nil, q: nil, date: nil, type: nil)
+      def list(limit: nil, before: nil, q: nil, date: nil, type: nil, include_pinned: true)
         return [] unless @session_manager
 
         live = @mutex.synchronize do
@@ -185,10 +196,26 @@ module Clacky
           }
         end
 
-        all = all.select { |s| (s[:created_at] || "") < before } if before
-        all = all.first(limit) if limit
+        # ── Split pinned vs non-pinned BEFORE applying `before`/`limit`.
+        # Pinned sessions bypass pagination entirely so an old pinned session
+        # never falls off the first page just because newer sessions exist.
+        # (Regression fix for 0.9.37: previously `all_sessions` was only
+        # sorted by created_at and `limit` cut off old pinned rows, making
+        # them invisible until the user clicked "load more".)
+        pinned, non_pinned = all.partition { |s| s[:pinned] }
+
+        # `before` cursor ONLY applies to non-pinned (paginated) sessions.
+        non_pinned = non_pinned.select { |s| (s[:created_at] || "") < before } if before
+        non_pinned = non_pinned.first(limit) if limit
+
+        # Pinned section: only included on the first page (before == nil) so
+        # "load more" responses don't re-send them. On first page, return ALL
+        # matching pinned sessions regardless of limit.
+        pinned_section = (include_pinned && before.nil?) ? pinned : []
+
+        ordered = pinned_section + non_pinned
 
-        all.map do |s|
+        ordered.map do |s|
           id = s[:session_id]
           ls = live[id]
           {

data/lib/clacky/tools/edit.rb

@@ -44,7 +44,10 @@ module Clacky
         end
 
         begin
-          content = File.read(path)
+          # Scrub invalid UTF-8 bytes at read time — otherwise editing a file
+          # that contains non-UTF-8 bytes would poison history / error messages
+          # and cause JSON.generate to fail during replay.
+          content = safe_utf8(File.read(path))
 
           # Find matching string using layered strategy (shared with preview)
           match_result = Utils::StringMatcher.find_match(content, old_string)
@@ -127,6 +130,13 @@ module Clacky
         replacements = result[:replacements] || result["replacements"] || 1
         "Modified #{replacements} occurrence#{replacements > 1 ? "s" : ""}"
       end
+
+      # Scrub invalid UTF-8 byte sequences (see file_reader.rb for rationale).
+      private def safe_utf8(str)
+        return str if str.nil?
+        return str if str.encoding == Encoding::UTF_8 && str.valid_encoding?
+        str.encode("UTF-8", invalid: :replace, undef: :replace, replace: "\u{FFFD}")
+      end
     end
   end
 end

data/lib/clacky/tools/file_reader.rb

@@ -86,8 +86,10 @@ module Clacky
             }
           end
 
-          # Read text file with optional line range
-          all_lines = File.readlines(expanded_path)
+          # Read text file with optional line range.
+          # Scrub invalid UTF-8 bytes (e.g. GBK-encoded files) so downstream
+          # JSON.generate / history persistence won't blow up later.
+          all_lines = File.readlines(expanded_path).map! { |line| safe_utf8(line) }
           total_lines = all_lines.size
 
           # Calculate start index (convert 1-indexed to 0-indexed)
@@ -313,7 +315,11 @@ module Clacky
       # List first-level directory contents (files and directories)
       private def list_directory_contents(path)
         begin
-          entries = Dir.entries(path).reject { |entry| entry == "." || entry == ".." }
+          # Scrub entry names — filenames on disk may contain non-UTF-8 bytes
+          # (e.g. GBK/Shift-JIS names on macOS/Linux) which would poison history.
+          entries = Dir.entries(path)
+                       .map { |entry| safe_utf8(entry) }
+                       .reject { |entry| entry == "." || entry == ".." }
 
           # Separate files and directories
           files = []
@@ -353,6 +359,16 @@ module Clacky
           }
         end
       end
+
+      # Scrub invalid UTF-8 byte sequences so the result survives
+      # JSON.generate (session replay, API responses).
+      # Invalid bytes are replaced with U+FFFD (�). Valid UTF-8 is
+      # returned untouched via the fast path.
+      private def safe_utf8(str)
+        return str if str.nil?
+        return str if str.encoding == Encoding::UTF_8 && str.valid_encoding?
+        str.encode("UTF-8", invalid: :replace, undef: :replace, replace: "\u{FFFD}")
+      end
     end
   end
 end

data/lib/clacky/tools/glob.rb

@@ -84,6 +84,7 @@ module Clacky
           always_ignored_dirs = Clacky::Utils::FileIgnoreHelper::ALWAYS_IGNORED_DIRS
 
           all_matches = Dir.glob(full_pattern, File::FNM_DOTMATCH)
+                           .map { |p| p.encoding == Encoding::UTF_8 && p.valid_encoding? ? p : p.encode("UTF-8", invalid: :replace, undef: :replace, replace: "\u{FFFD}") }
                            .reject { |path| File.directory?(path) }
                            .reject { |path| path.end_with?(".", "..") }
                            .reject do |path|

data/lib/clacky/tools/grep.rb

@@ -271,8 +271,10 @@ module Clacky
       def search_file(file, regex, context_lines, max_matches)
         matches = []
 
-        # Use File.foreach for memory-efficient line-by-line reading
-        File.foreach(file, chomp: true).with_index do |line, index|
+        # Use File.foreach for memory-efficient line-by-line reading.
+        # Scrub invalid UTF-8 bytes so results survive JSON encoding.
+        File.foreach(file, chomp: true).with_index do |raw_line, index|
+          line = safe_utf8(raw_line)
           # Stop if we have enough matches for this file
           break if matches.length >= max_matches
 
@@ -302,7 +304,7 @@ module Clacky
 
       # Get context lines around a match
       def get_line_context(file, match_index, context_lines)
-        lines = File.readlines(file, chomp: true)
+        lines = File.readlines(file, chomp: true).map! { |l| safe_utf8(l) }
         start_line = [0, match_index - context_lines].max
         end_line = [lines.length - 1, match_index + context_lines].min
 
@@ -325,6 +327,13 @@ module Clacky
       rescue StandardError
         nil
       end
+
+      # Scrub invalid UTF-8 byte sequences (see file_reader.rb for rationale).
+      private def safe_utf8(str)
+        return str if str.nil?
+        return str if str.encoding == Encoding::UTF_8 && str.valid_encoding?
+        str.encode("UTF-8", invalid: :replace, undef: :replace, replace: "\u{FFFD}")
+      end
     end
   end
 end

data/lib/clacky/tools/terminal.rb

@@ -147,6 +147,71 @@ module Clacky
         Clacky::Tools::Security.command_safe_for_auto_execution?(command)
       end
 
+      # ---------------------------------------------------------------------
+      # Internal Ruby API — synchronous capture
+      # ---------------------------------------------------------------------
+      #
+      # Run a shell command and BLOCK until it terminates, returning
+      # [output, exit_code]. Drop-in replacement for Open3.capture2e that
+      # goes through the same PTY + login-shell + Security pipeline used by
+      # the AI-facing tool (so rbenv/mise shims and gem mirrors work).
+      #
+      # Why this exists separately from #execute:
+      #
+      #   `execute` may return early with a :session_id the moment output
+      #   goes idle for DEFAULT_IDLE_MS (3s) — this is intentional for AI
+      #   agents (they can inspect progress, inject input, decide to kill).
+      #   Ruby callers like the HTTP server's upgrade flow only care about
+      #   "did it finish, with what output, what exit code" — they need
+      #   synchronous semantics. Previously each caller re-implemented the
+      #   poll loop (and 0.9.36's run_shell forgot to, causing the upgrade
+      #   failure bug).
+      #
+      # NOT exposed in tool_parameters — AI agents cannot invoke this.
+      #
+      # @param command [String]   the shell command to run
+      # @param timeout [Integer]  per-poll timeout AND the basis for the
+      #                           overall deadline (deadline = timeout + 60s)
+      # @param cwd     [String]   optional working directory
+      # @param env     [Hash]     optional env overrides
+      # @return [Array(String, Integer|nil)] [output, exit_code].
+      #         exit_code is nil only if the overall deadline was hit and
+      #         the session had to be force-killed.
+      def self.run_sync(command, timeout: 120, cwd: nil, env: nil)
+        terminal = new
+        result   = terminal.execute(
+          command: command,
+          timeout: timeout,
+          cwd:     cwd,
+          env:     env,
+        )
+        output   = result[:output].to_s
+
+        # Hard deadline in wall-clock terms — a genuinely stuck command
+        # must terminate. Each individual poll still carries `timeout`.
+        deadline = Time.now + timeout.to_i + 60
+
+        while result[:exit_code].nil? && result[:session_id] && Time.now < deadline
+          result = terminal.execute(
+            session_id: result[:session_id],
+            input:      "",
+            timeout:    timeout,
+          )
+          output += result[:output].to_s
+        end
+
+        # Deadline exceeded — best-effort cleanup so the session doesn't leak.
+        if result[:exit_code].nil? && result[:session_id]
+          begin
+            terminal.execute(session_id: result[:session_id], kill: true)
+          rescue StandardError
+            # swallow — cleanup is best-effort
+          end
+        end
+
+        [output, result[:exit_code]]
+      end
+
       # ---------------------------------------------------------------------
       # 1) Start a new command
       # ---------------------------------------------------------------------

data/lib/clacky/utils/model_pricing.rb

@@ -105,6 +105,42 @@ module Clacky
         }
       },
 
+      # DeepSeek V4 models
+      # Source: https://api-docs.deepseek.com/quick_start/pricing (USD / 1M tokens)
+      # DeepSeek billing model:
+      #   - "cache miss input" = regular prompt_tokens rate
+      #   - "cache hit input"  = cache_read rate (DeepSeek has no separate cache-write charge)
+      #   - No tiered pricing (single rate regardless of context length)
+      "deepseek-v4-flash" => {
+        input: {
+          default: 0.14,                  # $0.14/MTok cache miss
+          over_200k: 0.14                 # no tiered pricing
+        },
+        output: {
+          default: 0.28,                  # $0.28/MTok
+          over_200k: 0.28
+        },
+        cache: {
+          write: 0.14,                    # DeepSeek doesn't charge extra for writes; bill at miss rate
+          read: 0.028                     # $0.028/MTok cache hit
+        }
+      },
+
+      "deepseek-v4-pro" => {
+        input: {
+          default: 1.74,                  # $1.74/MTok cache miss
+          over_200k: 1.74
+        },
+        output: {
+          default: 3.48,                  # $3.48/MTok
+          over_200k: 3.48
+        },
+        cache: {
+          write: 1.74,                    # no separate write charge; bill at miss rate
+          read: 0.145                     # $0.145/MTok cache hit
+        }
+      },
+
       # Default fallback pricing (conservative estimates)
       "default" => {
         input: {
@@ -238,6 +274,15 @@ module Clacky
           "claude-3-5-sonnet-20240620"
         when /claude-3-5-haiku-20241022/i
           "claude-3-5-haiku-20241022"
+        when /deepseek-v4-pro/i, /deepseek.*v4.*pro/i
+          "deepseek-v4-pro"
+        when /deepseek-v4-flash/i, /deepseek.*v4.*flash/i
+          "deepseek-v4-flash"
+        # Legacy aliases: deepseek-chat and deepseek-reasoner are being
+        # deprecated on 2026-07-24 and map to deepseek-v4-flash's
+        # non-thinking / thinking modes respectively. Bill at flash rates.
+        when /^deepseek-chat$/i, /^deepseek-reasoner$/i
+          "deepseek-v4-flash"
         else
           "default"
         end

data/lib/clacky/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Clacky
-  VERSION = "0.9.36"
+  VERSION = "0.9.37"
 end

data/lib/clacky/web/sessions.js

@@ -1027,8 +1027,14 @@ const Sessions = (() => {
       Sessions.renderList();
 
       try {
-        // Cursor: oldest created_at in the current list
+        // Cursor: oldest created_at in the current list, EXCLUDING pinned
+        // sessions. The backend always returns ALL pinned sessions on the
+        // first page (they bypass pagination), so their created_at is
+        // irrelevant for cursor calculation. Including them here would
+        // cause the cursor to jump too far back and skip sessions between
+        // the oldest pinned one and the real last-loaded non-pinned row.
         const oldest = _sessions.reduce((min, s) => {
+          if (s.pinned) return min;                       // ignore pinned
           if (!s.created_at) return min;
           return (!min || s.created_at < min) ? s.created_at : min;
         }, null);

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: openclacky
 version: !ruby/object:Gem::Version
-  version: 0.9.36
+  version: 0.9.37
 platform: ruby
 authors:
 - windy