local-lemonade 1.0.0

package/README.md

@@ -0,0 +1,520 @@
+# local-lemonade
+
+A [pi](https://github.com/earendil-works/pi-mono) extension that connects pi to a
+self-hosted [Lemonade](https://github.com/lemonade-sdk/lemonade) server — a local
+LLM/audio/image server with GPU acceleration — and keeps that connection in sync
+with whatever the server is actually doing.
+
+No hardcoded endpoints, no manual model lists. One extension, four capabilities:
+
+1. **Live model discovery** — every chat model your lemonade box advertises shows
+   up in `/model` automatically, annotated with whether it's loaded right now.
+2. **Nine agent tools** — transcription, image generation/editing/upscaling,
+   text-to-speech, music/SFX, 3D meshes, and text classification, all callable
+   by the LLM, all model-aware.
+3. **A setup menu** — `/lemonade-setup` opens a navigable TUI for status,
+   discovery, configuration, model management (live-progress pulls, Hugging
+   Face installs), and live log streaming.
+4. **Harness self-awareness** — every chat, including virgin ones, is told
+   where its lemonade machinery lives, so the agent can operate it directly.
+
+---
+
+## Why this exists
+
+Lemonade is a *server*, and its contents change constantly: you load models, unload
+them, pull new ones. pi, by contrast, learns about models from a static file. The
+job of this extension is to erase that gap: every time pi starts, it asks lemonade
+what it currently has, and registers exactly that.
+
+## Design philosophy — why this is built pi-native
+
+This extension is not just a feature bridge; every structural choice reflects
+how pi itself works:
+
+- **Dynamic truth over static files.** pi learns providers from a static file;
+  this extension re-asks the server at every startup and re-registers, so
+  `/model` always mirrors reality — down to the `loaded` / `on-demand`
+  annotations, which come from live health state, never guesses.
+- **Native tools, not protocol shims.** Every capability is a plain
+  `pi.registerTool()` function with a `promptSnippet` — pi's own advertisement
+  mechanism, flattened by the harness into the provider's dialect. MCP appears
+  only where it's genuinely MCP: lemonade's own gateway ships as an optional,
+  disabled stdio entry for *other* MCP-consuming setups. Trusted in-process
+  code is never wrapped in JSON-RPC — that would add lifecycle overhead and
+  couple failure domains for zero benefit.
+- **Harness-level self-awareness.** The model has no inherent knowledge of the
+  machinery it runs on — the harness must *tell* it, in the one place it reads
+  every turn. The `promptGuidelines` lines inject the machinery's location
+  and the configured instance list into every chat's system prompt, so
+  "what's loaded on your server?" resolves deterministically in a virgin
+  chat instead of by lucky path-hunting.
+- **Advertisement that reflects reality.** Tool snippets use a `[lemonade]`
+  prefix (a corpus-bucket convention, so tools group visibly by origin in the
+  prompt), and `classify_text`'s description is *generated per session* from
+  the live catalog — the advertisement enumerates the classifiers that
+  actually exist on the box, and refreshes itself on every startup.
+- **Everything configurable, nothing hardcoded.** Every endpoint is a path key
+  in `lemonade.json`. Moving the server, changing ports, or tracking upstream
+  API changes is config, not code.
+- **Live state where it matters.** Pulls stream real progress from
+  server-owned download jobs with Esc-to-cancel; status shows host resources
+  and last-request performance; logs stream over websocket. Destructive
+  actions gate behind confirm dialogs and a typed phrase.
+- **Failures speak.** Missing models produce errors naming what to pull — the
+  agent relays and recovers. Never a silent failure.
+- **One entry point.** A single command — `/lemonade-setup` — every capability
+  navigable from one TUI menu; no command sprawl.
+- **Ground truth snapshotted.** `docs/` holds the official API spec so this
+  extension can be audited against it offline, and the behavior reference
+  records what was empirically confirmed on which server build.
+
+## Quick start
+
+1. Make sure your lemonade server is running and reachable.
+2. Create the config: copy the fully-commented `lemonade.example.json`
+   (in this folder) to `~/.pi/agent/lemonade.json` and point the first
+   `servers` entry's `baseUrl` at your server. This file is **required** —
+   the extension fails loudly and registers nothing without it, because
+   nothing is hardcoded.
+3. Start pi. Every chat-capable model on the server appears in `/model` as
+   `lemonade-<instance-name>/<model-id>` (e.g. `lemonade-main/Qwen3-8B-GGUF`),
+   labeled `loaded` or `on-demand`.
+4. Pick one and chat. Models marked *on-demand* are auto-loaded by lemonade when
+   first requested — if the target slot is occupied by a pinned model, lemonade
+   replies with a `slots_pinned_error` you'll see as an error message.
+5. Say *"transcribe `/path/to/file.mp3`"* in any session — the agent calls
+   `transcribe_audio` and Whisper returns the text.
+6. Type `/lemonade-setup` to manage everything interactively.
+
+## How the pieces work
+
+### Model discovery
+
+At every startup (and after `/reload`, and after any change made in the setup
+menu), the extension queries two endpoints:
+
+- `GET /v1/models` — the catalog. Each entry carries rich metadata:
+  `labels` (chat, reasoning, vision, transcription, image…), `recipe`,
+  `max_context_window`, `downloaded`, size.
+- `GET /api/v1/health` — live state: which models are *actually* loaded, per-type
+  slot limits, the realtime websocket port.
+
+It then registers one pi provider per configured instance, uniformly named
+`lemonade-<instance-name>`, with one pi model per catalog entry, mapped as
+follows:
+
+| Lemonade says | pi registers |
+|---|---|
+| `labels` contains `chat` (or `recipe: llamacpp`) | included (by default; see *Chat-only filter*) |
+| `labels` contains `reasoning` | `reasoning: true` |
+| `labels` contains `vision` | image input enabled |
+| `max_context_window` | context window |
+| model appears in `/health` | name suffix `(lemonade-<instance>, loaded)` |
+| not in `/health` | name suffix `(lemonade-<instance>, on-demand)` |
+
+If the server is unreachable at startup, the provider registers **nothing** —
+an unreachable model is not listable, so `/model` gets no lemonade entries
+until the box is reachable. (Consequence, accepted: sessions pinned to a
+lemonade-<name> model hard-fail model resolution while offline — "Model not found" —
+and recover on the next startup or `/lemonade-setup` → Refresh once the box
+is back.)
+
+**Chat-only filter** — lemonade also hosts transcription (Whisper), image
+generation (Flux), and other non-chat models. pi can only drive chat-completions
+models, so those are filtered out by default. Toggle via *Model management →
+Toggle chat-only filter* if you want them listed anyway.
+
+**Compat flags** — the provider is registered with lemonade-appropriate settings
+(`max_tokens` instead of `max_completion_tokens`, `system` instead of `developer`
+role, no `reasoning_effort`), so requests work out of the box.
+
+### Agent tools
+
+The extension registers nine tools the LLM can call on your behalf. Two facts
+apply to all of them:
+
+**Instance selection.** Every tool takes an optional `server` argument naming
+a configured instance (omit for the default). `"default"` is kept as a
+reserved alias; the default's actual name works too. Unknown names produce an
+error listing available instances. See *Multiple lemonade instances*.
+
+**Model selection precedence.** Tools that need a model resolve it in this
+order: (1) an explicit `model` argument in the tool call, (2) the
+extension's configured default (`defaultImageModel`, `defaultUpscaleModel`,
+`defaultTranscriptionModel`, `defaultClassifierModel` in `lemonade.json`),
+(3) auto-pick — the first catalog model carrying the right label (`image`,
+`edit`, `tts`, ...) or recipe (`onnxruntime` for classifiers). Set a default
+in `/lemonade-setup` or config if you want a specific model to win.
+
+**How results come back.** Text results (transcripts) are returned inline into
+the conversation. Binary outputs (images, audio, meshes) are *written to
+disk* — `outputDir` (default: the agent's working directory) as
+`lemonade-<kind>-<timestamp>.<ext>` — and the tool returns the saved path.
+The model relays the path to you. To *view* a generated image in the TUI, ask
+the agent to `read` the PNG afterwards: pi renders images inline in supported
+terminals (Kitty, iTerm2, Ghostty, WezTerm, Warp), and reading it also lets
+the model itself see and verify its own output. The TUI cannot render audio
+or `.glb` meshes at all — those live on disk only.
+
+Per tool:
+
+| Tool | What it does | Model resolution | Output | Caveats |
+|---|---|---|---|---|
+| `transcribe_audio` | Speech → text | `defaultTranscriptionModel` (Whisper-Large-v3) | transcript returned inline (not saved) | whisper.cpp can't demux containers — mp4/m4a/webm/... are converted to 16 kHz mono wav via **ffmpeg** on pi's machine first; raw wav/mp3 go straight through. Unloaded Whisper auto-loads into the separate transcription slot |
+| `generate_image` | Text → PNG | arg → `defaultImageModel` → first catalog model labeled `image` | PNG on disk + path | First call may include model load (observed ~12 s). Turbo models want `steps: 4`, `cfg_scale: 1`; the tool schema documents this for the model |
+| `edit_image` | Image + prompt → edited PNG | arg → `defaultImageModel` → first model labeled `edit` (then `image`) | PNG on disk + path | multipart upload of the source PNG; optional `mask_path` (white = edit, black = preserve). Edit-capable models on current build: the Flux family |
+| `vary_image` | Image → variation PNG | arg → `defaultImageModel` → first `edit`/`image` model | PNG on disk + path | no prompt parameter — variation is derived from the input image alone |
+| `upscale_image` | Image → 4x PNG | arg → `defaultUpscaleModel` (RealESRGAN-x4plus) | PNG on disk + path | needs an upscale model pulled on the server; `-anime` variant exists for art. Sends the image as base64 JSON, not multipart |
+| `text_to_speech` | Text → spoken audio | arg → first model labeled `tts` (OpenMOSS-TTS on current builds) | mp3/wav on disk + path | if the TTS model isn't pulled, the request can trigger a slow auto-pull that outlasts the request — pull it via the setup menu first. `voice`/`speed`/`response_format` supported |
+| `generate_audio` | Prompt → music / SFX | arg → first model labeled `music`/`sfx`/`audio` | wav on disk + path | ACE-Step for music (optional `lyrics` with `[verse]`/`[chorus]` tags = sung vocals; omit = instrumental), ThinkSound for SFX. Not downloaded by default — pull first |
+| `classify_text` | Text → ranked label confidences | arg → `defaultClassifierModel` → first catalog model with the `onnxruntime` recipe | inline text readout | encoder classifiers (phishing, PII, prompt-injection...) — millisecond inference in its own classification slot; scores returned highest-first, so the top line is the verdict. Pull classifiers via the setup menu or HF install |
+| `generate_3d_model` | Image → textured `.glb` | arg → first model labeled `3d` (TRELLIS-3D) | .glb on disk + path | minutes per mesh; `resolution` 512/1024/1536; needs TRELLIS pulled. TUI cannot render it — open the file in a 3D viewer |
+
+**How `classify_text` handles multiple classifiers and modalities.** Each
+classifier's label universe (phishing/benign, PII/no-PII, 1–5 stars…) is baked
+into the model at training time — the tool is a pipe, not a policy. Three
+layers make multi-classifier use workable: (1) the tool's prompt advertisement
+dynamically enumerates the classifiers actually on the server each session;
+(2) per-call selection via the `model` argument (or `defaultClassifierModel`
+config); (3) runtime self-description — the response's labels reveal the
+chosen model's universe, so a mismatched question ("is this a hot dog?" sent
+to a phishing model) returns an obviously wrong label set and the agent
+retries with the right model. Modality boundaries are architectural:
+`/v1/classify` serves *text encoders only* — image classification routes to
+vision-capable chat models (read the image), and audio classifies only after
+`transcribe_audio`. Sequence-classification models return the flat ranked
+scores; token-classification models (e.g. PII *span* detectors) return
+additional structure, which the tool passes through verbatim.
+
+Missing-model behavior is uniform: if no suitable model exists on the server,
+the tool returns a clear error naming what to pull — the agent relays it and
+suggests pulling via `/lemonade-setup`. Never a silent failure.
+
+Lemonade also exposes **realtime transcription** over a websocket
+(`ws://<host>:<ws-port>/realtime?model=…`; the port comes from `/health`).
+This extension does not wire that up — it's for live-mic streaming, not file
+transcription.
+
+### Agent self-awareness
+
+Every chat's system prompt — including virgin chats with no lemonade context —
+carries three guideline lines injected by this extension (via pi's
+`promptGuidelines` mechanism, attached to `transcribe_audio`):
+
+- where the machinery lives: `~/.pi/agent/lemonade.json` (base URL, endpoints,
+  defaults) and this README + `docs/` (full reference)
+- what to do with it: query the server's HTTP API directly via `bash` +
+  `curl` for state questions (loaded models, health, host resources), and
+  point the user at `/lemonade-setup` for interactive management
+- which instances exist: the configured fleet, by name and URL, and that
+  tools take an optional `server` argument (this line is generated per session)
+
+So asking the agent "what models are loaded on your server?" in a fresh chat
+resolves deterministically: read the config, curl `/v1/health`, summarize —
+no hunting for paths, no lucky guesses.
+
+### `/lemonade-setup` menu
+
+A navigable TUI (↑↓ / enter / esc):
+
+| Menu | What you can do |
+|---|---|
+| **Server status** | Live health: version, loaded models with type/device/pinned/context, per-type slot limits, websocket port — plus host resources (CPU/RAM/GPU/VRAM) and last-request performance (tok/s, time-to-first-token) |
+| **Server settings** | Discover servers (UDP beacon + HTTP fallback), edit base URL, edit API key, test connection |
+| **Model management** | List catalog (including not-yet-downloaded registry entries), load / unload, pull (download) with live progress + esc-to-cancel, install from Hugging Face (search → pick variant → install as `user.*`), delete, change context size (unload → reload with `ctx_size`, saved), toggle chat-only filter, refresh provider |
+| **Transcription settings** | Default Whisper model, endpoint path, smoke-test a file |
+| **Live server logs** | WebSocket stream of the server's log: full snapshot backlog (up to 5000 entries) then live entries as they happen; Esc to close |
+
+Notes on the destructive and slow paths:
+
+- **Delete** requires two gates: a confirm dialog, then typing the exact phrase
+  `i want to delete <model-id>` — three attempts allowed, each prompt shows
+  chances remaining, and Esc aborts immediately.
+- **Change context size** prefills the model's *current* context (from live
+  health), accepts `32k`, `1m`, or raw token counts, and warns that the model
+  is unloaded and reloaded (the new `ctx_size` is saved for future loads).
+- **Pull** uses lemonade's server-owned download jobs: a live progress view
+  (percent, bytes, per-file status, updated every second) with **Esc to cancel**
+  (via `/v1/downloads/control`). Blocking pull is kept only as a fallback for
+  older servers without job support.
+- **Install from Hugging Face** searches the registry through the server
+  (`/v1/registry/search`), lists quantization variants with sizes
+  (`/v1/pull/variants`), and installs the chosen one as a `user.*` model —
+  including vision/mmproj detection — with the same live progress view.
+
+Every change is saved to the config file and re-registers the provider
+immediately — no restart or `/reload` needed.
+
+**Server discovery** — lemonade broadcasts a JSON beacon
+(`{"service":"lemonade","hostname":…,"url":…}`) roughly once per second on UDP
+port 13305. The setup menu can listen for it, then falls back to probing common
+ports on localhost *and* the currently configured host.
+
+> **WSL2 note:** UDP LAN broadcasts do not propagate into WSL2's NAT'd virtual
+> network, so the beacon may never arrive if pi runs inside WSL2. The HTTP
+> fallback still works, and manual URL entry always works.
+
+## Configuration
+
+Everything lives in `~/.pi/agent/lemonade.json` — and it is **required**:
+the extension fails loudly at startup (red error in the chat window, nothing
+registered) if the file is missing or unparseable, because no endpoint is
+hardcoded and a config-less run would target a server that isn't yours.
+
+The tracked `lemonade.example.json` in this folder is the canonical
+reference: fully commented, documenting every key and its default, and
+doubly useful as a template:
+
+```bash
+cp ~/.pi/agent/extensions/local-lemonade/lemonade.example.json ~/.pi/agent/lemonade.json
+```
+
+Edit the first `servers` entry's `baseUrl`, trim the rest to taste, restart pi.
+`//` comments are allowed in the config file (the loader strips them);
+note that saving via `/lemonade-setup` rewrites the file and removes
+comments, so keep annotations in your copy of the example. Changes made in
+`/lemonade-setup` apply immediately — no restart or `/reload` needed.
+
+A minimal working config is one entry:
+
+```json
+{ "servers": [{ "name": "main", "baseUrl": "http://your-lemonade-server:13305" }] }
+```
+
+The full shape (values shown are the built-in defaults):
+
+```json
+{
+  "servers": [
+    { "name": "main", "baseUrl": "http://localhost:13305",
+      "description": "Primary lemonade server." }
+  ],
+  "apiKey": "lemonade",
+  "chatPath": "/api/v1",
+  "modelsPath": "/v1/models",
+  "healthPath": "/api/v1/health",
+  "loadPath": "/api/v1/load",
+  "unloadPath": "/api/v1/unload",
+  "pullPath": "/api/v1/pull",
+  "deletePath": "/api/v1/delete",
+  "downloadsPath": "/v1/downloads",
+  "downloadsControlPath": "/v1/downloads/control",
+  "registrySearchPath": "/v1/registry/search",
+  "pullVariantsPath": "/v1/pull/variants",
+  "transcriptionPath": "/v1/audio/transcriptions",
+  "imageGenerationPath": "/v1/images/generations",
+  "imageEditPath": "/v1/images/edits",
+  "imageVariationPath": "/v1/images/variations",
+  "imageUpscalePath": "/v1/images/upscale",
+  "speechPath": "/v1/audio/speech",
+  "audioGenerationPath": "/v1/audio/generations",
+  "mesh3dPath": "/v1/3d/generations",
+  "classifyPath": "/v1/classify",
+  "beaconPort": 13305,
+  "chatOnly": true,
+  "defaultTranscriptionModel": "Whisper-Large-v3",
+  "defaultImageModel": "",
+  "defaultUpscaleModel": "RealESRGAN-x4plus",
+  "defaultClassifierModel": "",
+  "outputDir": "",
+  "servers": [
+    { "name": "main", "baseUrl": "http://localhost:13305",
+      "description": "Primary lemonade server." }
+  ],
+  "discoveryTimeoutMs": 5000,
+  "beaconTimeoutMs": 3000,
+  "loadTimeoutMs": 300000,
+  "pullTimeoutMs": 1800000,
+  "transcriptionTimeoutMs": 300000,
+  "generationTimeoutMs": 600000
+}
+```
+
+| Key | Meaning |
+|---|---|
+| `servers` | **Required, non-empty** — the only instance store: `[{ "name", "baseUrl", "apiKey"?, "description"? }]`. The FIRST entry is the default instance (targeted when tools' `server` argument is omitted); reordering changes the default. Each registers as a `lemonade-<name>` provider. |
+| `apiKey` | Shared fallback `Authorization: Bearer …` for instances without their own. A dummy value works for auth-less servers (pi requires non-empty auth to list models). |
+| `*Path` | API endpoint paths, in case a future lemonade changes them. |
+| `chatOnly` | Filter the provider down to chat-capable models. |
+| `defaultTranscriptionModel` / `defaultImageModel` / `defaultUpscaleModel` / `defaultClassifierModel` | Fixed model ids for the tools. Empty = auto-pick from the live catalog (by label, or by `onnxruntime` recipe for classifiers). |
+| `outputDir` | Where generated files (images, audio, meshes) are saved. Empty = agent's working directory. |
+| `*TimeoutMs` | Per-operation timeouts. `pullTimeoutMs` only caps the *blocking* pull fallback — normal pulls use download jobs with live progress. |
+
+## Offline & unreachable networks
+
+When pi starts on a network with no route back to the lemonade LAN (different
+WAN, VPN down, box asleep):
+
+- **Startup never blocks longer than `discoveryTimeoutMs`** (~5 s by default —
+  lower it in config if you work offline often).
+- **Only the lemonade provider is affected** — every other provider in
+  `/model` (built-ins, other extensions) is untouched.
+- **No lemonade listings when unreachable.** If an instance can't be reached
+  at startup or refresh, it contributes nothing to `/model` — an unreachable
+  model is not listable. This is fully automated in both directions: entries
+  appear exactly when the box is reachable and disappear when it isn't
+  (until the next startup or Refresh re-checks).
+  **The accepted consequence:** sessions or scripts pinned to
+  `lemonade-<name>/<model>` hard-fail model resolution while offline
+  (`Error: Model "lemonade/<id>" not found`), and resume working once the box
+  is back and a Refresh has re-registered it. Non-reachable chat requests are
+  the honest outcome; phantom listings are never shown.
+- **Tools fail with actionable text, not cryptic errors**: every tool detects
+  the unreachable server and returns the same message — *where* it tried to
+  connect and *what to do about it* (reconnect, or point the extension at a
+  reachable instance via `/lemonade-setup`). No misleading "pull a model
+  first" advice when the real problem is the network, and no bare
+  `fetch failed`.
+- **The setup menu diagnoses it**: Server status shows an explicit
+  `Unreachable: <health URL>` panel, and Test connection reports the failure.
+- **The agent can self-diagnose**: the self-awareness guidelines point it at
+  the config, so asking the agent "are you connected to lemonade?" resolves
+  via a direct probe rather than guessing.
+
+**Levels of availability checking** (each with one job, each defined once):
+
+| Level | When | What it does | Where the logic lives |
+|---|---|---|---|
+| Registration probe | every startup / refresh | decides what goes into the provider: the live catalog, or nothing at all when unreachable — no phantom listings | `registerInstanceProvider` |
+| Tool fetch | every tool call | converts network rejections into the actionable unreachable message; user-cancels pass through | `toolFetch()` — used by all nine tools |
+| Auto-pick | model resolution | distinguishes unreachable (`null`) from no-match (`undefined`) so advice fits the situation | `pickModelByLabels` / `pickModelByRecipe` |
+| Instance resolution | every tool call with a `server` arg | resolves the named box's config view or returns an error listing known instances | `instanceView()` |
+
+Tools stay deliberately stateless — reachability is a per-call fact (networks
+change mid-session), so no connection state is cached between calls. The only
+repetition is the three-line binding guard at the top of each tool, which is
+what makes the rest of each tool body instance- and failure-aware for free.
+
+## Multiple lemonade instances
+
+The extension supports **simultaneous multi-instance use**. Each configured
+lemonade box becomes its own pi provider, every tool can target any instance,
+and the setup menu operates per-instance.
+
+**Configuration** — there is no separate default instance: `servers[]` is
+the *only* instance store, and the first entry IS the default. Every entry
+has a **name** (required) and an optional **description** (free-form context
+for your own reminder's sake). The shared top-level `apiKey` applies to any
+entry that doesn't define its own:
+
+```json
+{
+  "servers": [
+    { "name": "main", "baseUrl": "http://192.168.1.10:13305",
+      "description": "Primary lemonade server." },
+    { "name": "second-server", "baseUrl": "http://192.168.1.20:13305",
+      "description": "The other box, mostly for experiments." }
+  ]
+}
+```
+
+Names identify instances everywhere, without prejudice: provider ids, model
+addresses, and picker brackets are uniformly `lemonade-<name>` for EVERY
+instance — the default reads `[lemonade-main]` exactly like an extra
+reads `[lemonade-second-server]`. Descriptions are free-form context shown in the
+instance lists; edit both via *Manage instances → Edit instance*. Renaming
+any instance changes its provider id (and re-registers it), so pinned
+sessions and `--model lemonade-<old-name>/...` references must follow the
+rename — the cost of a perfectly uniform namespace.
+
+**How instances are exposed:**
+
+- **Providers:** every instance registers as `lemonade-<name>` — the default
+  included, no special case. Picker brackets, model addresses, and provider
+  column all read `lemonade-<name>` uniformly. Each is independently annotated `loaded` / `on-demand`
+  from that box's own health state. Unreachable boxes register nothing —
+  there are no offline placeholders for any instance.
+- **Tools:** every tool takes an optional `server` argument naming an
+  instance. Omitted → default. Unknown names return an error listing the
+  available instances — never a wrong-box dial. Model auto-pick, status, and
+  everything else resolve against the selected instance's catalog.
+- **Self-awareness:** the prompt guideline enumerates the configured
+  instances by name and URL every session, so virgin chats know the fleet
+  without hunting.
+- **Setup menu:** a *Switch instance* item (appears when extras exist) picks
+  which box Status, Model management, and Live logs act on; Server settings →
+  *Manage instances* lists (with live reachability), adds, and removes
+  instances; discovered servers can be registered as default or as a named
+  instance.
+
+**The one-place rule:** instance semantics are defined exactly once —
+`instanceView()` returns a *config view* (same settings, swapped
+baseUrl/apiKey) that flows through every existing helper unchanged, so tools,
+discovery, TUI actions, and error handling all inherit instance behavior
+without per-tool logic.
+
+## Reference documentation (API ground truth)
+
+This folder's [`docs/`](./docs) directory contains the complete official
+Lemonade endpoint specification, snapshotted from the upstream repo
+(`lemonade-sdk/lemonade`, `docs/api/`) on 2026-09-12 — the same markdown the
+[docs site](https://lemonade-server.ai/docs/api/) renders. It is the source of
+ground truth for every endpoint, parameter, and response shape this extension
+uses:
+
+| File | Contents |
+|---|---|
+| [`docs/README.md`](./docs/README.md) | Spec index and design philosophy |
+| [`docs/openai.md`](./docs/openai.md) | OpenAI-compatible surface: chat/completions, embeddings, audio (transcription, speech, generations), images (generation/edit/variation/upscale), realtime, `/v1/models` + labels taxonomy |
+| [`docs/lemonade.md`](./docs/lemonade.md) | Lemonade-specific surface: load/unload/pull/delete, download jobs + control, registry search + pull variants, health/stats/system-stats/system-info, classify, audio/3D generation, log streaming, job engine, install/uninstall |
+| [`docs/mcp.md`](./docs/mcp.md) | The MCP gateway (lemonade as an MCP server) |
+| [`docs/ollama.md`](./docs/ollama.md), [`docs/anthropic.md`](./docs/anthropic.md), [`docs/llamacpp.md`](./docs/llamacpp.md) | Other compatibility surfaces (rerank, slots, etc.) |
+
+When lemonade updates, re-sync with:
+
+```bash
+git clone --depth 1 --filter=blob:none --sparse https://github.com/lemonade-sdk/lemonade /tmp/lemonade \
+  && cd /tmp/lemonade && git sparse-checkout set docs/api \
+  && cp docs/api/*.md ~/.pi/agent/extensions/local-lemonade/docs/
+```
+
+## Lemonade behavior reference
+
+Empirically confirmed against lemonade 11.7.0 (cross-checked with the
+[official endpoint spec](https://lemonade-server.ai/docs/api/)):
+
+- **Auto-loading.** A request for an unloaded model triggers an automatic load
+  attempt. If the model's slot is free or holds an unpinned model, lemonade
+  evicts/loads and serves the request. If the slot is occupied by a *pinned*
+  model, you get `slots_pinned_error: "All loaded models of type … are pinned.
+  Unload a model first."` — unload or unpin via the setup menu, then retry.
+- **Per-type slots.** The server holds one slot per model type (llm,
+  transcription, image, embedding, …), so Whisper can be loaded alongside your
+  chat model.
+- **change-ctx.** Implemented as unload → reload with `ctx_size` +
+  `save_options: true`, which persists the choice for future loads.
+
+## Troubleshooting
+
+| Symptom | Fix |
+|---|---|
+| No `lemonade-<name>/*` models in `/model` | `/lemonade-setup` → Server settings → Test connection; check `baseUrl`; run Refresh |
+| Error: `slots_pinned_error` on first message | The pinned model occupies the slot — unload it (Model management), or select the loaded model |
+| `transcribe_audio` "not available" from the agent | Tools need a `promptSnippet` to appear in the system prompt — that's pi behavior, not a bug here; this extension sets it |
+| Transcription of `.mp4` fails with ffmpeg missing | Install ffmpeg on the machine running pi (container formats are converted locally before upload) |
+| Discovery finds nothing | Expected inside WSL2 (UDP broadcasts don't cross the NAT); set the URL manually |
+| Server unreachable at startup | The instance registers nothing — no lemonade models in `/model` until it's reachable; fix the URL and Refresh |
+| Pull seems to hang | It's downloading — the blocking pull path sends no progress. Large models take minutes; the `pullTimeoutMs` cap guarantees control returns |
+
+## MCP Gateway (lemonade as an MCP server)
+
+Lemonade exposes itself as a genuine MCP server at `POST /mcp`
+(Streamable HTTP), serving five tools: `lemonade_list_models`, `lemonade_chat`,
+`lemonade_transcribe_audio`, `lemonade_generate_image`, and `lemonade_omni`.
+
+pi's MCP manager only accepts *remote* MCP endpoints over HTTPS, so this
+setup ships a tiny local bridge instead:
+
+- `~/.pi/agent/bin/lemonade-mcp-proxy.mjs` — stdio→HTTP proxy: reads
+  newline-delimited JSON-RPC on stdin, forwards each message to the lemonade
+  gateway, writes responses to stdout (SSE frames unwrapped, session id
+  passed through).
+- `~/.pi/agent/mcps-local/lemonade/server.json` — MCP manager entry
+  (transport: stdio, connection: lazy, **disabled by default**).
+
+To use it: open `/mcp` in pi, pick **lemonade**, choose *Test or refresh*,
+inspect the tool manifest, select the tools you want, and enable the server.
+The first approved call starts the connection. Note the overlap: these MCP
+tools duplicate what this extension registers natively — enable them only if
+you want lemonade tools available in *other* MCP-consuming setups.

package/docs/README.md

@@ -0,0 +1,22 @@
+# Lemonade Endpoints Spec
+
+The Lemonade HTTP service provides a wide array of standards-compliant and custom endpoints.
+
+Our design philosophy is:
+
+1. Ensure that Lemonade works out-of-box with all popular local AI apps.
+2. Prioritize using a pre-existing standard for functionality whenever possible.
+3. Add sufficient custom functionality to enable developers to build highly polished experiences.
+
+This spec details all supported endpoints. It is organized into pages that correspond to  which organization (OpenAI, Ollama, Lemonade, etc.) defined the endpoints.
+
+| API | Description |
+|-----|-------------|
+| [OpenAI-Compatible API](./openai.md) | Start here for the main API surface used by most SDKs and clients. |
+| [Ollama-Compatible API](./ollama.md) | Use this if your client expects Ollama-style behavior and routes. |
+| [Anthropic-Compatible API](./anthropic.md) | Use this for clients built around Anthropic's message format. |
+| [MCP Gateway](./mcp.md) | Use this to expose Lemonade as a Model Context Protocol server (POST /mcp). |
+| [llama.cpp-Specific API](./llamacpp.md) | Reference for llama.cpp-specific compatibility and conventions. |
+| [Lemonade-Specific API](./lemonade.md) | Local-first API for managing lifecycle, configuration, backends, etc. |
+
+A running server also serves these pages itself: [`GET /v1/docs`](./lemonade.md#get-v1docs) lists what it has and [`GET /v1/docs/{page}`](./lemonade.md#get-v1docspage) returns one, so the reference always matches the installed version and stays available offline.

package/docs/anthropic.md

@@ -0,0 +1,11 @@
+# Anthropic-Compatible API
+
+Lemonade supports an initial Anthropic Messages compatibility endpoint for applications that call Claude-style APIs.
+
+| Endpoint | Status | Notes |
+|----------|--------|-------|
+| `POST /v1/messages` | Supported | Supports both streaming and non-streaming. Query params like `?beta=true` are accepted. |
+
+Current scope focuses on message generation parity for common fields (`model`, `messages`, `system`, `max_tokens`, `temperature`, `stream`, and basic `tools`). Unsupported or unimplemented Anthropic-specific fields are ignored and surfaced via warning logs/headers.
+
+Those limits apply only where Lemonade converts to and from the OpenAI shape. A cloud provider registered with `--wire-format anthropic` is relayed unconverted, so no field is dropped — see [Cloud Offload](../guide/configuration/cloud.md#providers-that-speak-the-anthropic-messages-format).