@duckmind/deepquark-darwin-arm64 0.9.78 → 0.9.81

package/.deepquark/skills/bundled/duckmind-multimodal/SKILL.md

@@ -0,0 +1,171 @@
+---
+name: duckmind-multimodal
+description: >
+  How to use DuckMind's multimodal API to send images, PDFs, audio, video, and generate images.
+  Use this skill whenever the user wants to interact with the DuckMind API for any multimodal task,
+  including: sending images to vision models, generating images from text prompts, processing PDF documents,
+  sending or receiving audio, or analyzing video content. Also trigger when the user mentions DuckMind
+  in combination with any media type, asks about DuckMind's image generation, PDF parsing, audio
+  transcription, or video analysis capabilities. Trigger even if the user just says "DuckMind" with
+  a file processing context, or asks how to send files/media to DuckMind models.
+---
+
+# DuckMind Multimodal API
+
+This skill covers how to use DuckMind's unified `/api/v1/chat/completions` endpoint for multimodal inputs and outputs: images, PDFs, audio, and video.
+
+## Authentication
+
+Before making any API call, retrieve the DuckMind API key from DeepQuark's auth store:
+
+```bash
+# With jq (preferred)
+DUCKMIND_API_KEY=$(jq -r '.duckmind.key' ~/.local/share/deepquark/auth.json)
+
+# With python3 (fallback)
+DUCKMIND_API_KEY=$(python3 -c "import json; d=json.load(open('$HOME/.local/share/deepquark/auth.json')); print(d['duckmind']['key'])")
+```
+
+Use `$DUCKMIND_API_KEY` (bash) or the extracted string as the Bearer token in all requests. Never hardcode the key.
+
+## Quick Reference: Content Types
+
+| Modality        | Content Type    | URL Support | Base64 Support |
+|-----------------|-----------------|-------------|----------------|
+| Image (input)   | `image_url`     | Yes         | Yes            |
+| Image (output)  | Set `modalities`| N/A         | Returned as b64|
+| PDF             | `file`          | Yes         | Yes            |
+| Audio (input)   | `input_audio`   | No          | Yes (required) |
+| Audio (output)  | Set `modalities`| N/A         | Streamed as b64|
+| Video           | `video_url`     | Provider-specific | Yes       |
+
+## Image Inputs
+
+Send images to vision-capable models. Place text prompt first, then images. Supports `image/png`, `image/jpeg`, `image/webp`, `image/gif`.
+
+**Via URL:**
+```js
+content: [
+  { type: "text", text: "What's in this image?" },
+  { type: "image_url", image_url: { url: "https://example.com/photo.jpg" } }
+]
+```
+
+**Via Base64:**
+```js
+const base64 = `data:image/jpeg;base64,${Buffer.from(fs.readFileSync(path)).toString("base64")}`;
+// Then use: { type: "image_url", image_url: { url: base64 } }
+```
+
+Multiple images can be sent as separate entries in the content array.
+
+## Image Generation
+
+Models with `"image"` in `output_modalities` can generate images. Set the `modalities` parameter:
+
+- Text + image models (e.g. Gemini): `modalities: ["image", "text"]`
+- Image-only models (e.g. Flux): `modalities: ["image"]`
+
+```js
+const result = await fetch("https://openrouter.ai/api/v1/chat/completions", {
+  method: "POST",
+  headers: { Authorization: `Bearer ${DUCKMIND_API_KEY}`, "Content-Type": "application/json" },
+  body: JSON.stringify({
+    model: "google/gemini-2.5-flash-image-preview",
+    messages: [{ role: "user", content: "Generate a sunset over mountains" }],
+    modalities: ["image", "text"],
+    stream: false,
+  }),
+});
+```
+
+Response contains `choices[0].message.images[]` with base64 data URLs.
+
+**Image config options** (via `image_config` parameter):
+- `aspect_ratio`: "1:1" (default), "2:3", "3:2", "3:4", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9"
+  - Extended (Gemini 3.1 Flash only): "1:4", "4:1", "1:8", "8:1"
+- `image_size`: "1K" (default), "2K", "4K", "0.5K" (Gemini 3.1 Flash only)
+- `font_inputs`: Sourceful models only — custom font rendering, max 2, +$0.03 each
+- `super_resolution_references`: Sourceful models only — enhance low-quality elements, max 4, +$0.20 each
+
+For full details on image config options, see `references/image-generation.md`.
+
+## PDF Processing
+
+Use the `file` content type. Works on **any** DuckMind model — native models get direct passthrough, others get parsed content.
+
+```js
+content: [
+  { type: "text", text: "Summarize this document." },
+  {
+    type: "file",
+    file: { filename: "doc.pdf", file_data: "https://example.com/doc.pdf" }
+    // Or for base64: file_data: "data:application/pdf;base64,..."
+  }
+]
+```
+
+**PDF Engines** (via `plugins` parameter):
+- `"native"` — model's built-in file processing (charged as input tokens)
+- `"pdf-text"` — free, best for well-structured text PDFs
+- `"mistral-ocr"` — $2/1000 pages, best for scanned docs or PDFs with images
+
+Default: native if supported, otherwise `mistral-ocr`.
+
+```js
+plugins: [{ id: "file-parser", pdf: { engine: "pdf-text" } }]
+```
+
+**Skip re-parsing costs**: Store `annotations` from the response and include them in follow-up requests. See `references/pdf-processing.md` for the full annotation reuse pattern.
+
+## Audio
+
+### Audio Input
+Audio must be **base64-encoded** (URLs not supported). Use `input_audio` content type.
+
+```js
+content: [
+  { type: "text", text: "Transcribe this audio." },
+  { type: "input_audio", input_audio: { data: base64Audio, format: "wav" } }
+]
+```
+
+Supported formats: wav, mp3, aiff, aac, ogg, flac, m4a, pcm16, pcm24 (varies by model).
+
+### Audio Output
+Set `modalities: ["text", "audio"]` and provide `audio` config. **Requires streaming.**
+
+```js
+{
+  modalities: ["text", "audio"],
+  audio: { voice: "alloy", format: "wav" },
+  stream: true
+}
+```
+
+Voices: alloy, echo, fable, onyx, nova, shimmer (varies by model).
+Formats: wav, mp3, flac, opus, pcm16 (varies by model).
+
+Audio data arrives in `delta.audio.data` (base64 chunks) and `delta.audio.transcript`.
+
+## Video Input
+
+Use `video_url` content type. Supports base64 (`data:video/mp4;base64,...`) and provider-specific URLs (e.g. YouTube links for Gemini on AI Studio).
+
+## Model Compatibility
+
+Not all models support all modalities. Use the [Models page](https://openrouter.ai/models) to filter by:
+- Vision models → image input
+- File-compatible models → PDF processing
+- Audio-capable models → audio I/O
+- Video-capable models → video input
+- Output modalities with "image" → image generation
+
+## SDK Usage
+
+DuckMind provides official SDKs:
+- **TypeScript**: `@openrouter/sdk` — use `openRouter.chat.send()`
+- **Python**: `openrouter` package
+
+Both SDKs use camelCase for parameters (e.g. `imageUrl`, `inputAudio`, `fileData`).
+Raw fetch/requests use snake_case (e.g. `image_url`, `input_audio`, `file_data`).

package/.deepquark/skills/bundled/duckmind-multimodal/references/image-generation.md

@@ -0,0 +1,131 @@
+# Image Generation — Detailed Reference
+
+## Model Discovery
+
+Find image generation models:
+1. **Models page**: Filter by output modalities → "image"
+2. **Chatroom**: Click the "Image" button to auto-filter
+3. **API**: Check `output_modalities` includes `"image"`
+
+## Compatible Models (examples)
+
+- `google/gemini-3.1-flash-image-preview` (extended aspect ratios, 0.5K support)
+- `google/gemini-2.5-flash-image-preview`
+- `google/gemini-3-pro-image-preview`
+- `black-forest-labs/flux.2-pro`
+- `black-forest-labs/flux.2-flex`
+- `sourceful/riverflow-v2-standard-preview`
+- `sourceful/riverflow-v2-fast`
+- `sourceful/riverflow-v2-pro`
+
+## Modalities Parameter
+
+- Models outputting both text and images: `modalities: ["image", "text"]`
+- Image-only models (Flux, Sourceful): `modalities: ["image"]`
+
+## Image Config Options
+
+### Aspect Ratio (`image_config.aspect_ratio`)
+
+Standard ratios (all models):
+
+| Ratio | Resolution    |
+|-------|--------------|
+| 1:1   | 1024×1024 (default) |
+| 2:3   | 832×1248     |
+| 3:2   | 1248×832     |
+| 3:4   | 864×1184     |
+| 4:3   | 1184×864     |
+| 4:5   | 896×1152     |
+| 5:4   | 1152×896     |
+| 9:16  | 768×1344     |
+| 16:9  | 1344×768     |
+| 21:9  | 1536×672     |
+
+Extended ratios (Gemini 3.1 Flash only):
+
+| Ratio | Use Case                              |
+|-------|---------------------------------------|
+| 1:4   | Scrolling carousels, vertical UI      |
+| 4:1   | Hero banners, horizontal layouts      |
+| 1:8   | Notification headers, narrow vertical |
+| 8:1   | Wide-format banners, panoramic        |
+
+### Image Size (`image_config.image_size`)
+
+| Size | Description                          |
+|------|--------------------------------------|
+| 1K   | Standard resolution (default)        |
+| 2K   | Higher resolution                    |
+| 4K   | Highest resolution                   |
+| 0.5K | Lower resolution, efficient (Gemini 3.1 Flash only) |
+
+### Font Inputs (`image_config.font_inputs`) — Sourceful only
+
+Render custom text with specific fonts. Max 2 font inputs, +$0.03 each.
+
+```json
+{
+  "image_config": {
+    "font_inputs": [
+      { "font_url": "https://example.com/fonts/custom.ttf", "text": "Hello World" }
+    ]
+  }
+}
+```
+
+Tips:
+- Include the text in your prompt with font name, color, size, position details
+- `text` should match exactly what's in the prompt (no extra wording or quotes)
+- Use line breaks or double spaces to separate headlines and sub-headers
+- Works best with short, clear headlines
+
+### Super Resolution References (`image_config.super_resolution_references`) — Sourceful only
+
+Enhance low-quality elements using high-quality reference images. Only works with image-to-image (input images in messages). Max 4 references, +$0.20 each.
+
+```json
+{
+  "image_config": {
+    "super_resolution_references": [
+      "https://example.com/ref1.jpg",
+      "https://example.com/ref2.jpg"
+    ]
+  }
+}
+```
+
+Tips:
+- Output matches input image size — use larger inputs for better quality
+- Use high-quality references showing desired enhancement result
+
+## Streaming Image Generation
+
+Works with `stream: true`. Images arrive in `delta.images[]` within SSE chunks:
+
+```js
+// In each SSE chunk:
+parsed.choices[0].delta.images?.forEach(img => {
+  console.log(img.image_url.url); // base64 data URL
+});
+```
+
+## Response Format
+
+```json
+{
+  "choices": [{
+    "message": {
+      "role": "assistant",
+      "content": "Description text...",
+      "images": [{
+        "type": "image_url",
+        "image_url": { "url": "data:image/png;base64,..." }
+      }]
+    }
+  }]
+}
+```
+
+- Format: base64-encoded data URLs, typically PNG
+- Some models can generate multiple images per response

package/.deepquark/skills/bundled/duckmind-multimodal/references/pdf-processing.md

@@ -0,0 +1,120 @@
+# PDF Processing — Detailed Reference
+
+## How It Works
+
+PDFs are sent via the `file` content type in the messages array. DuckMind handles routing:
+- **Native support**: PDF passed directly to the model
+- **No native support**: DuckMind parses the PDF and passes extracted content to the model
+
+This means PDF processing works with **any** model on DuckMind.
+
+## PDF Engines
+
+Configure via the `plugins` parameter:
+
+```js
+plugins: [{ id: "file-parser", pdf: { engine: "pdf-text" } }]
+```
+
+| Engine        | Cost              | Best For                          |
+|---------------|-------------------|-----------------------------------|
+| `native`      | Input token cost  | Models with built-in file support |
+| `pdf-text`    | Free              | Well-structured text PDFs         |
+| `mistral-ocr` | $2 / 1,000 pages | Scanned docs, PDFs with images    |
+
+**Default behavior**: If no engine is specified, DuckMind uses `native` first (if available), then falls back to `mistral-ocr`.
+
+## Sending PDFs
+
+### Via URL (publicly accessible)
+```js
+{
+  type: "file",
+  file: {
+    filename: "document.pdf",
+    file_data: "https://example.com/document.pdf"  // SDK: fileData
+  }
+}
+```
+
+### Via Base64 (local/private files)
+```js
+const base64PDF = `data:application/pdf;base64,${Buffer.from(fs.readFileSync(path)).toString("base64")}`;
+
+{
+  type: "file",
+  file: {
+    filename: "document.pdf",
+    file_data: base64PDF  // SDK: fileData
+  }
+}
+```
+
+## Reusing Parsed Results (Skip Re-parsing Costs)
+
+When DuckMind parses a PDF, the response includes `annotations` in the assistant message. Include these annotations in follow-up requests to avoid re-parsing.
+
+### Step-by-step:
+
+1. **First request**: Send PDF normally
+2. **Extract annotations**: `response.choices[0].message.annotations`
+3. **Follow-up requests**: Include the annotations in the assistant message
+
+```js
+// Follow-up request structure:
+messages: [
+  // Original user message with PDF
+  { role: "user", content: [{ type: "text", text: "..." }, { type: "file", file: { ... } }] },
+  // Assistant response WITH annotations
+  { role: "assistant", content: "Previous response...", annotations: fileAnnotations },
+  // New user question
+  { role: "user", content: "Follow-up question about the document" }
+]
+```
+
+### File Annotation Schema
+
+```typescript
+type FileAnnotation = {
+  type: "file";
+  file: {
+    hash: string;           // Unique hash for cache matching
+    name?: string;          // Original filename
+    content: ContentPart[]; // Parsed content
+  };
+};
+
+type ContentPart =
+  | { type: "text"; text: string }
+  | { type: "image_url"; image_url: { url: string } };
+```
+
+The `content` array contains extracted text blocks and images (as base64 data URLs). The `hash` field is what DuckMind uses to skip re-parsing.
+
+## Response Format
+
+```json
+{
+  "choices": [{
+    "message": {
+      "role": "assistant",
+      "content": "The document discusses...",
+      "annotations": [{
+        "type": "file",
+        "file": {
+          "hash": "abc123...",
+          "name": "document.pdf",
+          "content": [
+            { "type": "text", "text": "Parsed text..." },
+            { "type": "image_url", "image_url": { "url": "data:image/png;base64,..." } }
+          ]
+        }
+      }]
+    }
+  }]
+}
+```
+
+## Multiple Files
+
+You can send both PDFs and other file types in the same request by adding multiple `file` entries to the content array.

package/.deepquark/skills/bundled/duckmind-transcribe/SKILL.md

@@ -0,0 +1,70 @@
+---
+name: duckmind-transcribe
+description: Transcribe audio files using OpenRouter API with audio-capable models (Gemini, GPT-4o-audio, etc). Use this skill whenever the user asks to transcribe audio, convert speech to text, get a transcript from an audio file, or work with audio recordings (.m4a, .mp3, .wav, .ogg, .flac, .aac, .wma, .opus). Also trigger when the user mentions voice memos, meeting recordings, podcast transcriptions, interview audio, or any task involving extracting text from audio content. Requires OPENROUTER_API_KEY environment variable and ffmpeg installed on the system.
+---
+# OpenRouter Audio Transcription
+Transcribe audio files via OpenRouter chat completions API using the input_audio content type. Compatible with any audio-capable model on OpenRouter (Gemini, GPT-4o-audio, etc).
+## Prerequisites
+Before running the transcription script, verify these dependencies are available:
+```bash
+for cmd in curl ffmpeg base64 jq; do
+  command -v "$cmd" >/dev/null 2>&1 || echo "Missing: $cmd"
+done
+```
+The OPENROUTER_API_KEY environment variable must be set. If not available, retrieve it from DeepQuark's auth store (see **API Key** section below).
+## API Key
+Before making any API call, retrieve the DuckMind API key from DeepQuark's auth store:
+```bash
+# With jq (preferred)
+DUCKMIND_API_KEY=$(jq -r '.duckmind.key' ~/.local/share/deepquark/auth.json)
+# With python3 (fallback)
+DUCKMIND_API_KEY=$(python3 -c "import json; d=json.load(open('$HOME/.local/share/deepquark/auth.json')); print(d['duckmind']['key'])")
+```
+Use `$DUCKMIND_API_KEY` (bash) or the extracted string as the Bearer token in all requests. Never hardcode the key.
+## Quick Start
+The main script is at {baseDir}/scripts/transcribe.sh. Make it executable first:
+```bash
+chmod +x {baseDir}/scripts/transcribe.sh
+```
+Basic transcription:
+```bash
+{baseDir}/scripts/transcribe.sh /path/to/audio.m4a
+```
+Output goes to stdout by default.
+## Options
+| Flag | Description | Default |
+|------|-------------|---------|
+| --model <model> | OpenRouter model ID | google/gemini-2.5-flash |
+| --prompt <text> | Custom transcription instructions | Standard transcription prompt |
+| --out <file> | Save transcript to file | stdout |
+| --title <name> | Caller ID for OpenRouter dashboard | Duckmina |
+| --referer <url> | HTTP-Referer header | https://duckmind.ai |
+## Usage Examples
+```bash
+# Transcribe with a different model
+{baseDir}/scripts/transcribe.sh audio.ogg --model openai/gpt-4o-audio-preview
+# Custom instructions (speaker labels, timestamps)
+{baseDir}/scripts/transcribe.sh meeting.m4a --prompt "Transcribe with speaker labels and timestamps"
+# Save output to a file
+{baseDir}/scripts/transcribe.sh interview.m4a --out /tmp/transcript.txt
+# Combine options
+{baseDir}/scripts/transcribe.sh podcast.mp3 --model openai/gpt-4o-audio-preview --prompt "Include timestamps" --out transcript.md
+```
+## How It Works
+1. Audio conversion: ffmpeg converts the input to WAV (mono, 16kHz) for consistent API compatibility
+2. Base64 encoding: The WAV file is base64-encoded
+3. API request: Sends the audio as input_audio content to OpenRouter chat completions endpoint
+4. Extraction: Pulls the transcript text from the API response
+## Workflow
+When a user asks to transcribe audio:
+1. Check that the audio file exists (look in /mnt/user-data/uploads/ for uploaded files)
+2. Retrieve `DUCKMIND_API_KEY` from `~/.local/share/deepquark/auth.json` (see **API Key** section). If the file is missing or the key is empty, ask the user
+3. Verify ffmpeg is installed. If not, try `apt-get install -y ffmpeg`
+4. Run the transcription script with appropriate options
+5. If the user wants the transcript saved, use --out or redirect stdout to a file
+6. Present the transcript to the user, and optionally save to /mnt/user-data/outputs/
+## Troubleshooting
+- ffmpeg format errors: The script uses a temp directory (not mktemp -t file.wav) because macOS mktemp adds random suffixes after the extension, breaking format detection
+- Argument list too long: Large audio files produce huge base64 strings. The script handles this by writing to temp files instead of passing data as CLI arguments
+- Empty response: Usually caused by an invalid API key, unsupported model, or corrupted audio. The script dumps the raw response for debugging
+- Model does not support audio: Switch to a known audio-capable model like google/gemini-2.5-flash or openai/gpt-4o-audio-preview

package/.deepquark/skills/bundled/duckmind-transcribe/scripts/transcribe.sh

@@ -0,0 +1,134 @@
+#!/usr/bin/env bash
+set -euo pipefail
+usage() {
+  cat >&2 <<'EOF'
+Usage:
+  transcribe.sh <audio-file> [options]
+Options:
+  --model <model>    OpenRouter model (default: google/gemini-2.5-flash)
+  --prompt <text>    Custom transcription instructions
+  --out <file>       Output file (default: stdout)
+  --title <name>     Caller identifier for OpenRouter (default: DuckMind)
+  --referer <url>    HTTP-Referer header (default: https://duckmind.ai)
+Example:
+  transcribe.sh audio.m4a --prompt "Include timestamps"
+EOF
+  exit 2
+}
+if [[ "${1:-}" == "" || "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
+  usage
+fi
+in="${1:-}"
+shift || true
+model="google/gemini-2.5-flash"
+prompt="Please transcribe this audio file. Keep it readable, suitable for messaging. Begin transcript immediately without any commentary."
+out=""
+title="DuckMind"
+referer="https://duckmind.ai"
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --model)
+      model="${2:-}"
+      shift 2
+      ;;
+    --prompt)
+      prompt="${2:-}"
+      shift 2
+      ;;
+    --out)
+      out="${2:-}"
+      shift 2
+      ;;
+    --title)
+      title="${2:-}"
+      shift 2
+      ;;
+    --referer)
+      referer="${2:-}"
+      shift 2
+      ;;
+    *)
+      echo "Unknown arg: $1" >&2
+      usage
+      ;;
+  esac
+done
+if [[ ! -f "$in" ]]; then
+  echo "File not found: $in" >&2
+  exit 1
+fi
+
+# Retrieve DuckMind API key from DeepQuark's auth store
+auth_file="$HOME/.local/share/deepquark/auth.json"
+if [[ ! -f "$auth_file" ]]; then
+  echo "Auth file not found: $auth_file" >&2
+  exit 1
+fi
+
+if command -v jq &>/dev/null; then
+  OPENROUTER_API_KEY=$(jq -r '.duckmind.key' "$auth_file")
+else
+  OPENROUTER_API_KEY=$(python3 -c "import json; d=json.load(open('$auth_file')); print(d['duckmind']['key'])")
+fi
+
+if [[ -z "${OPENROUTER_API_KEY:-}" || "$OPENROUTER_API_KEY" == "null" ]]; then
+  echo "Missing or invalid DuckMind API key in $auth_file" >&2
+  exit 1
+fi
+
+# Create temp files for conversion
+tmp_dir=$(mktemp -d)
+tmp_wav="$tmp_dir/audio.wav"
+trap 'rm -rf "$tmp_dir"' EXIT
+# Convert audio to WAV (mono, 16kHz) using ffmpeg
+if ! ffmpeg -y -i "$in" -ac 1 -ar 16000 "$tmp_wav" 2>/dev/null; then
+  echo "ffmpeg failed to convert audio" >&2
+  exit 1
+fi
+# Base64 encode the WAV file
+audio_base64=$(base64 < "$tmp_wav" | tr -d '\n')
+# Build JSON payload using jq for proper escaping
+# Write base64 to file first to avoid "argument list too long" errors
+echo "$audio_base64" > "$tmp_dir/audio.b64"
+payload_file="$tmp_dir/payload.json"
+jq -n \
+  --arg model "$model" \
+  --arg prompt "$prompt" \
+  --rawfile audio "$tmp_dir/audio.b64" \
+  '{
+    model: $model,
+    messages: [{
+      role: "user",
+      content: [
+        { type: "text", text: $prompt },
+        { type: "input_audio", input_audio: { data: ($audio | rtrimstr("\n")), format: "wav" } }
+      ]
+    }]
+  }' > "$payload_file"
+# Make API request
+response=$(curl -sS "https://openrouter.ai/api/v1/chat/completions" \
+  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
+  -H "Content-Type: application/json" \
+  -H "X-Title: $title" \
+  -H "HTTP-Referer: $referer" \
+  -d "@$payload_file")
+# Check for API errors first
+if echo "$response" | jq -e '.error' >/dev/null 2>&1; then
+  error_msg=$(echo "$response" | jq -r '.error.message // .error // "Unknown API error"')
+  echo "API error: $error_msg" >&2
+  exit 1
+fi
+# Extract transcript
+transcript=$(echo "$response" | jq -r '.choices[0].message.content // empty')
+if [[ -z "$transcript" ]]; then
+  echo "Empty response from API. Raw response:" >&2
+  echo "$response" | head -c 500 >&2
+  exit 1
+fi
+if [[ "$out" != "" ]]; then
+  mkdir -p "$(dirname "$out")"
+  echo "$transcript" > "$out"
+  echo "$out"
+else
+  echo "$transcript"
+fi