openclacky 1.3.2 → 1.3.3

data/lib/clacky/default_skills/extend-openclacky/SKILL.md

@@ -1,11 +1,11 @@
 ---
 name: extend-openclacky
-description: Customize, fix, override or extend openclacky itself — e.g. change a built-in tool's behavior, intercept/audit/block tool calls with shell scripts, or plug in a new IM channel (Slack, in-house IM, etc.). Trigger on phrases like "patch clacky", "patch openclacky", "change WebSearch behavior", "block dangerous commands", "audit tool use", "add Slack channel", "改 openclacky 内置", "改 clacky 内置", "monkey patch openclacky", "拦截工具调用". Do NOT trigger for ordinary feature work in the user's own project that doesn't touch openclacky.
+description: Customize, fix, override or extend openclacky itself — change a built-in tool's behavior, intercept/audit/block tool calls, plug in a new IM channel (Slack, in-house IM…), or add UI to the Web UI (panel, button, settings tab). Trigger on "patch openclacky", "block dangerous commands", "audit tool use", "add Slack channel", "extend the web ui", "改 openclacky 内置", "拦截工具调用", "扩展 web 界面". Do NOT trigger for ordinary feature work in the user's own project that doesn't touch openclacky.
 ---
 
 # Extending Openclacky
 
-Openclacky ships three official extension mechanisms that survive `gem update` and never require editing the gem source.
+Openclacky ships four official extension mechanisms that survive `gem update` and never require editing the gem source.
 **Never tell the user to `bundle show openclacky` and edit the gem — always use one of these.**
 
 ## Pick the right mechanism
@@ -15,6 +15,7 @@ Openclacky ships three official extension mechanisms that survive `gem update` a
 | Change behavior of an **existing method** in openclacky (e.g. `WebSearch#execute` timeout, fix a bug in a built-in tool) | **Patch** | `clacky patch_new <id> "Const#method" -d "<desc>"` | `clacky patch_verify` |
 | **Audit / block / observe** tool calls (block `rm -rf /`, log every shell command) — no Ruby needed | **Shell Hook** | `clacky hook_new <id> -e <event>` | `clacky hook_verify` |
 | Plug openclacky into a **new IM platform** (Slack, in-house IM, custom webhook…) | **Channel Adapter** | `clacky channel_new <platform_id>` | `clacky channel_verify` |
+| Add UI to the **Web UI** (custom panel, header button, settings tab, visualize data) | **Web UI Extension** | drop a `.js` file in `~/.clacky/webui_ext/` | reload page; `Clacky.ext.slots()` in console; `?pure=true` to escape |
 
 ## Authoritative documentation
 
@@ -23,14 +24,15 @@ Each mechanism has a full reference doc — read the relevant one with `web_fetc
 - Patches → https://www.openclacky.com/docs/extend-patches
 - Shell Hooks → https://www.openclacky.com/docs/extend-shell-hooks
 - Channel Adapters → https://www.openclacky.com/docs/extend-channel-adapter
+- Web UI Extensions → https://www.openclacky.com/docs/extend-webui
 
 ## Execution playbook
 
 1. **Identify** which mechanism fits (use the table above; ask if genuinely ambiguous).
 2. **Read the doc** for that mechanism with `web_fetch`. Don't guess fields, hook events, or required methods — the doc is the contract.
-3. **Run the scaffold** CLI command. It generates the file(s) in `~/.clacky/...` with correct meta.
+3. **Run the scaffold** CLI command. It generates the file(s) in `~/.clacky/...` with correct meta. *(Web UI Extensions have no scaffold — just create a `.js` file under `~/.clacky/webui_ext/`; the doc shows the `Clacky.ext` contract.)*
 4. **Edit** the generated file to implement the user's intent. Keep generated meta fields (`target`, `event`, `platform_id`, the `Clacky::ChannelRegistry.register(...)` line, etc.) intact unless the doc says otherwise.
-5. **Verify** with the matching `*_verify` command. Surface any `[FAIL]` lines to the user verbatim.
+5. **Verify** with the matching `*_verify` command. Surface any `[FAIL]` lines to the user verbatim. *(Web UI Extensions have no verify command — reload the page and confirm the slot rendered; if anything breaks, `?pure=true` disables all extensions instantly.)*
 
 ## When NOT to use this skill
 

data/lib/clacky/default_skills/media-gen/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: media-gen
-description: 'Generate images, videos, or audio (text-to-speech) in the current task. Use whenever the user asks to create/generate/produce a picture / image / illustration / cover / poster / icon / artwork, a video / clip / animation, or speech / voiceover / narration / TTS — e.g. 生成图片, 画一张, 做封面, 配图, generate image, make a picture, draw, design a cover, 生成视频, 做个视频, text-to-video, 朗读, 配音, 旁白, 文字转语音, generate speech, voiceover. Also use when a document (slides, poster, README hero) needs an inline image.'
+description: 'Generate or edit images, videos, or audio (text-to-speech) in the current task. Use whenever the user asks to create/generate/produce or edit/modify a picture / image / illustration / cover / poster / icon / artwork, a video / clip / animation, or speech / voiceover / narration / TTS — e.g. generate image, draw, design a cover, edit this image, change the background, text-to-video, generate speech; 画一张, 配图, 编辑图片, 改图, 换背景, 做个视频, 配音, 文字转语音. Also use when a document (slides, poster, README hero) needs an inline image.'
 disable-model-invocation: false
 user-invocable: true
 always-show: true
@@ -8,7 +8,7 @@ always-show: true
 
 # media-gen
 
-Generate images on demand by calling the local Clacky HTTP server, which dispatches to whichever image-generation model the user configured (`type=image` in their model settings).
+Generate **and edit** images on demand by calling the local Clacky HTTP server, which dispatches to whichever image-generation model the user configured (`type=image` in their model settings). Editing (image-in → image-out) works with any image model that accepts image input — most current ones do.
 
 ## Endpoint
 
@@ -73,7 +73,32 @@ curl -s -X POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/ima
 |----------------|----------|-------------------------------------|-------|
 | `prompt`       | yes      | string                              | Be detailed and concrete. See prompt tips below. |
 | `aspect_ratio` | no       | `landscape` / `square` / `portrait` | Defaults to `landscape`. |
-| `output_dir`   | no       | absolute path                       | Defaults to the current working directory. The image is saved under `<output_dir>/assets/generated/`. |
+| `output_dir`   | no       | absolute path                       | Per-call override. When omitted, falls back to the user's `media_output_dir` setting (Settings → Models → Media Output Directory), then to the working directory. The image is always saved into the `assets/generated/` subdirectory of whichever path is used. |
+| `image`        | no       | file path / base64 / data URL       | A single input image to **edit**. Triggers image-edit mode (see below). |
+| `images`       | no       | array of the above                  | Multiple input images for a multi-image edit. Takes precedence over `image`. |
+
+### Editing an existing image
+
+To edit instead of generate from scratch, pass the existing image as `image`
+(a local file path is easiest — the skill reads and encodes it for you) plus a
+`prompt` describing the change. The configured image model receives the
+image alongside the prompt and returns an edited result.
+
+```bash
+curl -s -X POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/image \
+  -H "Content-Type: application/json" \
+  -d '{
+    "prompt": "change the background to a starry night sky, keep the cat unchanged",
+    "image": "/abs/path/to/input.png"
+  }'
+```
+
+- The result is a **new** edited image saved under `assets/generated/` — the
+  original file is never modified in place.
+- For combining several inputs (e.g. "put the product from image 1 onto the
+  background from image 2"), pass them as `images: ["/path/a.png", "/path/b.png"]`
+  and describe the composition in the prompt.
+- Same speed/concurrency rules apply: editing is as slow as generation, one at a time.
 
 ### Response shape (success)
 
@@ -81,7 +106,7 @@ curl -s -X POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/ima
 {
   "success": true,
   "image": "/abs/path/to/working_dir/assets/generated/img_20260525_011820_a1b2c3d4.png",
-  "model": "or-gemini-3-pro-image",
+  "model": "<the configured image model>",
   "provider": "openclacky",
   "prompt": "A clean, modern hero illustration ...",
   "aspect_ratio": "landscape",
@@ -145,7 +170,6 @@ When the user gives a vague request like "给我配张图", ask one clarifying q
 
 ## When NOT to use this skill
 
-- The user asks to **edit** an existing image (this skill is text-to-image only today)
 - The user wants a **diagram / chart** with specific data — use a charting library (matplotlib, mermaid, etc.) instead; image gen is for illustrations, not data viz
 - The user asks for **screenshots** of real software — use the browser tool
 
@@ -190,7 +214,7 @@ curl -s -X POST http://${CLACKY_SERVER_HOST}:${CLACKY_SERVER_PORT}/api/media/vid
 | `aspect_ratio`     | no       | `landscape` / `portrait`        | Defaults to `landscape` (16:9). |
 | `duration_seconds` | no       | 4–8                             | Defaults to 8. |
 | `image`            | no       | `{ "b64_json": "...", "mime_type": "image/png" }` | Optional first frame for image-to-video. |
-| `output_dir`       | no       | absolute path                   | MP4 saved under `<output_dir>/assets/generated/`. |
+| `output_dir`       | no       | absolute path                   | Per-call override; same fallback chain as `image` (user setting → cwd). MP4 saved into the `assets/generated/` subdirectory of whichever path is used. |
 
 ### Response (success)
 

data/lib/clacky/media/openai_compat.rb

@@ -2,6 +2,7 @@
 
 require "faraday"
 require "json"
+require "base64"
 require_relative "base"
 
 module Clacky
@@ -28,7 +29,7 @@ module Clacky
       VIDEO_ASPECTS = %w[landscape portrait].freeze
       DEFAULT_VIDEO_DURATION = 8
 
-      def generate_image(prompt:, aspect_ratio: DEFAULT_ASPECT, output_dir: nil, n: 1, **_kwargs)
+      def generate_image(prompt:, aspect_ratio: DEFAULT_ASPECT, output_dir: nil, n: 1, image: nil, images: nil, **_kwargs)
         provider_id = Clacky::Providers.find_by_base_url(@base_url) || "custom"
         aspect      = ASPECT_TO_SIZE.key?(aspect_ratio) ? aspect_ratio : DEFAULT_ASPECT
         size        = ASPECT_TO_SIZE[aspect]
@@ -52,6 +53,18 @@ module Clacky
           )
         end
 
+        begin
+          input_images = normalize_input_images(image, images)
+        rescue ArgumentError => e
+          return error_response(
+            error: e.message,
+            error_type: "invalid_argument",
+            provider: provider_id,
+            prompt: prompt,
+            aspect_ratio: aspect
+          )
+        end
+
         payload = { model: @model, n: n }
         if gemini_family?(@model)
           # Gemini image models (routed via openclacky / openrouter gateway)
@@ -64,6 +77,11 @@ module Clacky
           payload[:size]   = size
         end
 
+        # With input image(s) this becomes an edit: the gateway forwards them
+        # to the model alongside the prompt. Sent as `images` (array) so
+        # multi-image edits work; the gateway also accepts a single `image`.
+        payload[:images] = input_images unless input_images.empty?
+
         begin
           response = connection.post("images/generations") do |req|
             req.headers["Content-Type"]  = "application/json"
@@ -306,6 +324,51 @@ module Clacky
         model_name.to_s.match?(/gemini|imagen/i)
       end
 
+      # Normalise the optional image/edit inputs into an array of data URLs
+      # ("data:<mime>;base64,<payload>") the gateway understands. Each input
+      # may be a local file path, a data URL, or a bare base64 string.
+      # `images` (array or single) takes precedence over `image`.
+      # Raises ArgumentError on a missing file or undecodable input.
+      private def normalize_input_images(image, images)
+        raw = images.nil? ? image : images
+        return [] if raw.nil?
+
+        list = raw.is_a?(Array) ? raw : [raw]
+        list.filter_map do |item|
+          s = item.to_s.strip
+          next if s.empty?
+          to_data_url(s)
+        end
+      end
+
+      private def to_data_url(input)
+        return input if input.start_with?("data:")
+
+        # A filesystem path → read and encode.
+        if File.file?(input)
+          bytes = File.binread(input)
+          mime  = mime_for_path(input)
+          return "data:#{mime};base64,#{Base64.strict_encode64(bytes)}"
+        end
+
+        # Otherwise treat as a bare base64 payload; validate it decodes.
+        begin
+          Base64.strict_decode64(input)
+        rescue ArgumentError
+          raise ArgumentError, "input image is neither an existing file path nor valid base64"
+        end
+        "data:image/png;base64,#{input}"
+      end
+
+      private def mime_for_path(path)
+        case File.extname(path).downcase
+        when ".jpg", ".jpeg" then "image/jpeg"
+        when ".webp"         then "image/webp"
+        when ".gif"          then "image/gif"
+        else                      "image/png"
+        end
+      end
+
       # base_url is taken verbatim from PRESETS (each provider already
       # includes the API version segment when needed). We only ensure a
       # trailing slash so Faraday's relative-path join behaves.

data/lib/clacky/media/output_dir.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+module Clacky
+  module Media
+    # Resolves the on-disk root for generated media files (images, videos,
+    # audio) according to a fixed precedence:
+    #
+    #   1. `param`      — explicit `output_dir` from the API caller.
+    #                     Highest priority; lets a single call land
+    #                     somewhere specific (e.g. a doc's project root).
+    #   2. `configured` — user setting from AgentConfig#media_output_dir.
+    #                     Set via Settings → Models → Media Output Directory.
+    #   3. `fallback`   — process default; preserves legacy behavior for
+    #                     configs that have neither key set.
+    #
+    # Pure function on purpose: callers (HTTP handlers) read the configured
+    # value off AgentConfig and inject it here. Keeps this helper trivially
+    # unit-testable and free of global state.
+    #
+    # The final on-disk path is `<resolved>/assets/generated/<file>` —
+    # the `assets/generated/` suffix is appended by Media::Base#save_*
+    # for stable relative-path semantics across markdown / slide outputs,
+    # and is intentionally not configurable here.
+    module OutputDir
+      # @param param [String, nil]      explicit per-call override
+      # @param configured [String, nil] user-configured default
+      # @param fallback [String]        last-resort default (defaults to Dir.pwd)
+      # @return [String]                absolute or `~`-prefixed path; the
+      #   caller's File.join with "assets/generated/" handles `~` via the
+      #   surrounding FileUtils.mkdir_p call only when expanded — for safety
+      #   we expand `~` here so downstream sees an absolute path.
+      def self.resolve(param:, configured:, fallback: Dir.pwd)
+        chosen = first_present(param, configured) || fallback
+        File.expand_path(chosen.to_s)
+      end
+
+      # @api private
+      def self.first_present(*candidates)
+        candidates.find { |c| c.is_a?(String) && !c.strip.empty? }
+      end
+    end
+  end
+end

data/lib/clacky/message_history.rb

@@ -85,6 +85,15 @@ module Clacky
       self
     end
 
+    # Truncate history starting from the user message with the given created_at timestamp.
+    # Removes that message and everything after it. Returns self.
+    def truncate_from_created_at(created_at)
+      idx = @messages.index { |m| m[:role] == "user" && m[:created_at].to_s == created_at.to_s }
+      return self unless idx
+
+      truncate_from(idx)
+    end
+
     # Roll back the history to just before the given message object.
     # Removes the message and anything appended after it.
     # Used to undo a failed speculative append (e.g. compression message that errored).

data/lib/clacky/server/channel/channel_manager.rb

@@ -43,6 +43,8 @@ module Clacky
         @running           = false
         @mutex             = Mutex.new
         @session_counters  = Hash.new(0)
+        @dedup_mutex       = Mutex.new
+        @last_message      = {}  # channel_key => [digest, monotonic_time]
       end
 
       # Start all enabled adapters in background threads. Non-blocking.
@@ -242,6 +244,16 @@ module Clacky
         files = event[:files] || []
         return if (text.nil? || text.empty?) && files.empty?
 
+        # Safety net against adapter-side message storms: drop an identical message
+        # repeated on the same channel within a short window. A real user never
+        # sends the exact same text+files twice within DEDUP_WINDOW seconds, but a
+        # misbehaving adapter (or noisy IM system messages) can, which would
+        # otherwise spin the interrupt→restart loop below indefinitely.
+        if duplicate_message?(event, text, files)
+          Clacky::Logger.info("[ChannelManager] Dropping duplicate message on #{channel_key(event)} within #{DEDUP_WINDOW}s")
+          return
+        end
+
         # Handle built-in commands
         if text&.match?(KNOWN_COMMAND) || text&.match?(/\A([\?h]|help)\z/i)
           handle_command(adapter, event, text)
@@ -749,6 +761,20 @@ module Clacky
         adapter.send_text(chat_id, "Recent sessions:\n#{lines.join("\n")}\n\nUse `/bind <n>` to switch.")
       end
 
+      DEDUP_WINDOW = 2.0  # seconds; identical messages on the same channel within this window are dropped
+
+      private def duplicate_message?(event, text, files)
+        key    = channel_key(event)
+        digest = [text, files.map { |f| f.is_a?(Hash) ? f[:name] : f.to_s }].hash
+        now    = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
+        @dedup_mutex.synchronize do
+          prev_digest, prev_time = @last_message[key]
+          @last_message[key] = [digest, now]
+          !prev_digest.nil? && prev_digest == digest && (now - prev_time) < DEDUP_WINDOW
+        end
+      end
+
       def channel_key(event)
         platform = event[:platform].to_s
         case @binding_mode

data/lib/clacky/server/git_panel.rb

@@ -0,0 +1,115 @@
+# frozen_string_literal: true
+
+require "open3"
+
+module Clacky
+  module Server
+    # Read-mostly git operations scoped to a session's working directory, backing
+    # the official "git" WebUI panel. Commands run with explicit argv (no shell),
+    # so user-supplied values (paths, messages) cannot inject. Write operations
+    # are limited to a guarded `commit`; history-rewriting / remote-mutating
+    # commands are never exposed here.
+    module GitPanel
+      module_function
+
+      # Run a git subcommand in `dir` with argv-style args (no shell). Returns
+      # [stdout, stderr, success_bool]. Never raises on git failure.
+      def git(dir, *args)
+        out, err, status = Open3.capture3("git", "-C", dir.to_s, *args)
+        [out, err, status.success?]
+      rescue StandardError => e
+        ["", e.message, false]
+      end
+
+      # Whether `dir` is inside a git work tree.
+      def repo?(dir)
+        out, _err, ok = git(dir, "rev-parse", "--is-inside-work-tree")
+        ok && out.strip == "true"
+      end
+
+      # { branch:, ahead:, behind:, files: [{ path:, x:, y:, staged:, untracked: }] }
+      # Parsed from `git status --porcelain=v2 --branch`.
+      def status(dir)
+        out, _err, ok = git(dir, "status", "--porcelain=v2", "--branch")
+        return { branch: nil, files: [] } unless ok
+
+        branch = nil
+        ahead = behind = 0
+        files = []
+        out.each_line do |line|
+          line = line.chomp
+          if line.start_with?("# branch.head ")
+            branch = line.sub("# branch.head ", "")
+          elsif line.start_with?("# branch.ab ")
+            m = line.match(/\+(\d+) -(\d+)/)
+            ahead, behind = m[1].to_i, m[2].to_i if m
+          elsif line.start_with?("1 ", "2 ")
+            xy = line.split(" ")[1]
+            path = line.split(" ", 9).last
+            files << { path: path, x: xy[0], y: xy[1],
+                       staged: xy[0] != ".", untracked: false }
+          elsif line.start_with?("? ")
+            files << { path: line.sub("? ", ""), x: "?", y: "?",
+                       staged: false, untracked: true }
+          end
+        end
+        { branch: branch, ahead: ahead, behind: behind, files: files }
+      end
+
+      # Unified diff. `file` (optional, relative) limits to one path; omitted =
+      # whole working tree (tracked changes). `--` guards path from being read
+      # as an option.
+      def diff(dir, file: nil)
+        args = ["diff"]
+        args += ["--", file] if file && !file.empty?
+        out, _err, _ok = git(dir, *args)
+        out
+      end
+
+      # Recent commits: [{ hash:, short:, author:, date:, subject: }].
+      def log(dir, limit: 50)
+        limit = limit.to_i.clamp(1, 200)
+        fmt = "%H%x1f%h%x1f%an%x1f%ad%x1f%s"
+        out, _err, ok = git(dir, "log", "-n", limit.to_s, "--date=short", "--pretty=format:#{fmt}")
+        return [] unless ok
+
+        out.each_line.filter_map do |line|
+          h, short, author, date, subject = line.chomp.split("\x1f")
+          next unless h
+          { hash: h, short: short, author: author, date: date, subject: subject }
+        end
+      end
+
+      # [{ name:, current: bool }] from `git branch`.
+      def branches(dir)
+        out, _err, ok = git(dir, "branch", "--format=%(refname:short)%00%(HEAD)")
+        return [] unless ok
+
+        out.each_line.filter_map do |line|
+          name, head = line.chomp.split("\x00")
+          next unless name && !name.empty?
+          { name: name, current: head == "*" }
+        end
+      end
+
+      # Stage `files` (relative paths) and commit with `message`. Returns
+      # { ok:, error?:, hash? }. Refuses empty message / empty file set. Uses
+      # argv so paths/message cannot inject; no --no-verify, no amend.
+      def commit(dir, message:, files:)
+        msg   = message.to_s.strip
+        paths = Array(files).map(&:to_s).reject(&:empty?)
+        return { ok: false, error: "commit message is required" } if msg.empty?
+        return { ok: false, error: "no files selected" } if paths.empty?
+
+        _out, add_err, add_ok = git(dir, "add", "--", *paths)
+        return { ok: false, error: "git add failed: #{add_err.strip}" } unless add_ok
+
+        _out, c_err, c_ok = git(dir, "commit", "-m", msg, "--", *paths)
+        return { ok: false, error: "git commit failed: #{c_err.strip}" } unless c_ok
+
+        head, _err, _ok = git(dir, "rev-parse", "--short", "HEAD")
+        { ok: true, hash: head.strip }
+      end
+    end
+  end
+end