@nodetool-ai/agents 0.8.4-a24.3 → 0.8.4

package/dist/_skills/nodetool-js-scripting/SKILL.md

@@ -0,0 +1,223 @@
+---
+name: nodetool-js-scripting
+description: "Write JavaScript that runs in NodeTool's QuickJS sandbox: Code node bodies, saved JS script documents, sandbox package imports, and calling nodes from code."
+featured: true
+---
+
+# Write NodeTool sandbox JavaScript
+
+Three places take the same guest JavaScript, with the same globals, the same
+imports and the same limits.
+
+| Where | What it is | Reach it with |
+|---|---|---|
+| **Code node** | A body inside a workflow graph | `validate_code`, `run_code`, `test_code` |
+| **JS script document** | A named, versioned script with declared ports, secrets, a timeout and saved tests | `list_js_scripts`, `get_js_script`, `save_js_script`, `edit_js_script`, `validate_js_script`, `run_js_script`, `test_js_script` |
+| **Native flow** | Guest code calling nodes as async functions, no graph | `import "@nodetool-ai/sandbox-nodetool/flow"` |
+
+Pick a Code node when the logic belongs to one graph. Pick a script document
+when the same logic is called from several places, needs its own tests, or needs
+version history. Pick native flow when the control flow is easier to write than
+to wire.
+
+## The body contract
+
+```js
+// inputs: { rows: [...], threshold: 10 }
+const kept = inputs.rows.filter((r) => r.score > inputs.threshold);
+progress(50, `kept ${kept.length}`);
+for (const row of kept) await emit("row", row);   // streams as it goes
+await output("count", kept.length);               // final, posted at the end
+```
+
+- **Inputs arrive on the `inputs` object**, never as globals of their own name.
+  Values are deep-copied through JSON before entering the guest.
+- **Outputs leave through `await output(name, value)` and `await emit(name,
+  value)`, and nothing else.** `name` must be a declared output handle. A second
+  `output` on the same handle throws. `return` is ordinary control flow and its
+  value is ignored. For a script document this is enforced: a body that declares
+  outputs and returns them instead of emitting them is an **error**, because a
+  script has no legacy return contract. A Code node still accepts the old
+  return/yield contract for one more release, with a deprecation warning.
+- **Media inputs arrive as refs.** Read one with `media.bytes` / `media.text` /
+  `media.info`, and build one with `media.toDocument` / `toImage` / `toAudio` /
+  `toVideo`. Pass the whole input object, not its `uri`, so the ref's own type
+  travels with it. Every `media.*` call is async.
+- **`stream` is the input side of `emit`.** A body that mentions `stream` runs
+  **once** over the whole stream and pulls its own items. A body that never
+  mentions it runs once per incoming item. Nothing is configured: deleting the
+  last `stream` call flips the node back.
+
+```js
+for await (const item of stream(name))            // one handle, in order, until EOS
+for await (const [handle, item] of stream.any())  // every handle, arrival order
+const item = await stream.first(name)             // next value, undefined at EOS
+stream.open(name)                                 // could more still arrive?
+```
+
+- **`state`** survives across streaming invocations and resets at each run.
+- **`progress(percent, message)`** drives the node's progress bar.
+
+## Capabilities are globals, libraries are imports
+
+Globals need no declaration: `console`, `fetch`, `workspace`, `getSecret`,
+`nodetool.secrets.*`, `sleep`, `crypto`, `format`, `image`, `audio`, `video`,
+`media`, `canvas`, `assetToSandbox` / `sandboxToAsset`, `progress`, `emit` /
+`output`, and the pure helpers `toBase64` / `fromBase64` / `toHex` / `fromHex` /
+`parallelMap`. `media.*` and `workspace.*` need a `ProcessingContext`, so they
+throw in a bare sandbox call with no run behind it.
+
+Everything else is a **sandbox package** the body imports:
+
+```js
+import yaml from "@nodetool-ai/sandbox-yaml";
+```
+
+Discover them with `list_sandbox_packages` and read one's API with
+`get_sandbox_package_docs`. Do not guess a specifier: a pack this host does not
+carry fails validation with "Install `<pack>`". `nodetool packs
+compile` builds one for a dependency that is not shipped.
+
+`image.*`, `audio.*` and `video.*` transforms return a `sandbox://media/<id>`
+handle, not bytes. Chain the calls, then `<type>.toAsset(handle)` before the run
+ends. A handle does not survive into a later run.
+
+## Running an agent from code
+
+Use `run_agent` when the work needs more than one model call: look something
+up, act, read the result, correct.
+
+```js
+import { run_agent } from "@nodetool-ai/sandbox-nodetool/agents";
+
+const answer = await run_agent({
+  prompt: "Read storyboard " + inputs.storyboardId + " and list its shot titles.",
+  model: inputs.model,                // {provider, id} from a model selector
+  tools: ["get_storyboard"],          // capability wire names, nothing else
+  output_schema: { type: "object", properties: { titles: { type: "array", items: { type: "string" } } } }
+});
+await output("titles", answer.result.titles);
+```
+
+Without `output_schema`, `answer.text` is the agent's final message. The agent
+reports its text and tool calls while it runs. A mini app shows them in an
+Agent Activity widget bound to `op:<id>/exec#transcript`. `generate_text` stays
+the right call for one prompt with no tools.
+
+## Calling nodes from code
+
+```js
+import "@nodetool-ai/sandbox-nodetool/flow";   // mounts the bridge, required
+import { concat } from "@nodetool-ai/sandbox-flow/nodetool.text";
+
+const r = await concat({ a: inputs.left, b: inputs.right });
+await output("joined", r.output);
+```
+
+`await` is the edge, a variable is the wire, `Promise.all` is the fan-out. Both
+imports are required: the facade does not mount without the capability module.
+Streaming-output nodes carry `.stream(inputs)`, an async iterable where an early
+`break` closes the stream and runs node cleanup. Every call passes the per-call
+permission gate and bills through the invoking run, bounded by a recursion depth
+cap of 4 and 16 concurrently open streams per run. Streaming *inputs* accept
+arrays only in v1.
+
+A program that must open in the editor, be validated, or run on the server still
+builds a graph. Use `nodetool-workflow-builder`
+for that.
+
+## Building a timeline from code
+
+```js
+import { video } from "@nodetool-ai/sandbox-timeline";
+
+const v = video({ width: 1920, height: 1080, fps: 30 });
+const title = v.scene("title", 3, (s) => {
+  const t = s.text("Hello", { size: 120, weight: 600, color: "#ffffff" });
+  t.enter({ from: { opacity: 0 }, at: 0, dur: 0.5 });
+});
+v.series([title]);
+const saved = await v.save(nodetool.timelines, { name: "Hello" });
+await output("timeline_id", saved.timeline_id);
+```
+
+The pack authors a whole motion-graphics cut in seconds, local to each scene,
+and saves it with one `set_timeline_document`, instead of one `edit_timeline`
+op per clip. Pass the body's `nodetool.timelines` to `v.save`: a module cannot
+see the belt. `get_sandbox_package_docs` on the specifier returns the full
+authoring API.
+
+Save the build script itself as a JS script document and it links to the
+timeline it makes automatically: `v.save(nodetool.timelines, {name})` with no
+`timeline_id` the first time creates the timeline and links this script to
+it; every later run of the same script updates that timeline in place
+instead of making another. Revise with `edit_js_script`'s string-replacement
+ops (below) rather than resending the whole body.
+
+## The loop
+
+1. **Write or read the body.** `get_js_script` for a saved script.
+2. **`validate_code` / `validate_js_script` after every edit.** It catches
+   syntax, imports against the installed catalog, undefined names, undeclared
+   `inputs.*` reads, outputs no `emit`/`output` call reaches, duplicate or
+   non-identifier port names, and tests naming ports the script does not
+   declare. It is far cheaper than running.
+3. **`run_code` / `run_js_script`** with real inputs. `output` values come back
+   as `outputs`, `emit` values as `streamed`, an ordered list of `{name, value}`.
+   Stage a streamed input with `input_streams`, keyed by handle.
+4. **`test_code` / `test_js_script`** as the regression check. A case supplies
+   `inputs` (or `input_streams`) and optionally `expect` — final values per
+   handle, compared structurally, with unnamed outputs ignored, and
+   `expected_streamed`, the full ordered emit list. A case with neither passes
+   when the body runs without error.
+5. **Save.** `save_js_script` validates first and is CAS on update. For a
+   small revision, `edit_js_script(js_script_id, ops)` is cheaper: each op is
+   `{old, new}`, and `old` must match exactly once in the current code — like
+   a code editor's find/replace, refused by name on zero or on an ambiguous
+   match. It validates the result the same way `save_js_script` does.
+
+Script documents get the same version family as the other documents:
+`list_js_script_versions`, `get_js_script_version`, `create_js_script_version`,
+`restore_js_script_version`, `delete_js_script_version`.
+
+## Secrets and limits
+
+Read a credential with `nodetool.secrets.get(name)` — never inline one, and
+never write one into a body or a test case. A script's declared secrets are
+intersected with whatever allowance the invoking context carries. There is no
+`set_secret`: `request_secret` asks the user's own client for one and the value
+never enters the guest, the transcript, or the model's context. A headless run
+carries no secret prompt, so the call is refused by name rather than quietly
+writing something nobody approved.
+
+Script composition is bounded like sub-agents: depth cap 4 with a script id
+chain, so a cycle fails the call naming it. Execution time defaults to 30s and
+every limit is overridable per invocation and clamped to a ceiling.
+
+## Verify from a shell
+
+These are the `nodetool` CLI. From a NodeTool checkout the same commands run
+as `npm run dev:nodetool -- <command>`.
+
+```bash
+nodetool jsscript validate <id|file.json> --json
+nodetool jsscript run <id|file.json> --inputs '{"numbers":[1,2,3]}'
+nodetool jsscript run <id|file.json> --input-streams '{"numbers":[1,2,3]}'
+nodetool jsscript test <id|file.json> --json
+nodetool jsscript versions list|show|create|restore|delete <id>
+```
+
+A path that exists on disk wins over an id, and a file target needs no database.
+`jsscript test` exits non-zero on any failure and is the keyless selfcheck the
+harness gate runs.
+
+## Reference
+
+In a NodeTool checkout, these repository sources go further:
+
+- `docs/javascript-sandbox.md` — the guest surface, marshaling, limits,
+  concurrency and the security model.
+- `docs/js-script-document-design.md` — document shape, storage, invocation
+  from agents, Code nodes and mini apps.
+- `docs/harnesses.md` § nodetool jsscript
+- `packages/sandbox-packs/README.md`

package/dist/_skills/nodetool-model-provider-config/SKILL.md

@@ -0,0 +1,348 @@
+---
+name: nodetool-model-provider-config
+description: "Configure NodeTool model providers and API keys (list_secrets, request_secret), pick a model with find_model, or set up local Ollama, llama.cpp and Hugging Face models."
+featured: true
+---
+
+You help users configure AI model providers and select the right models for their tasks.
+
+# In the product
+
+The agent has no shell. Configure keys and pick models with these tools:
+
+1. `list_secrets` shows which credentials are configured. It never returns a
+   value.
+2. `request_secret {key, reason, help_url}` opens a dialog where the user types
+   a missing key. The value is saved from the dialog and never passes through
+   the agent. Never ask the user to paste a key into chat. When the run has no
+   interactive client, name the missing key and point the user to
+   Settings > Secrets.
+3. `find_model {capability}` returns a real `{provider, model_id}` and a `ref`
+   to assign verbatim to a node's model property. Call it before adding any
+   generic AI node. Never write a model id from memory, including the examples
+   below.
+4. `list_provider_models {provider}` lists one provider's language models.
+
+Inside a code action the same calls are `nodetool.secrets` /
+`nodetool.settings` (load `api-settings`) and `nodetool.models` (load
+`api-models`).
+
+The `nodetool` provider is NodeTool's managed service. Its models run on
+platform keys and draw from the user's credit balance, so it needs no key of
+the user's own. Every other provider bills the user's own account.
+
+# Provider Overview
+
+| Provider | Type | Key Env Var | Models |
+|----------|------|-------------|--------|
+| **OpenAI** | Cloud | `OPENAI_API_KEY` | GPT, GPT-Image, TTS, Whisper |
+| **Anthropic** | Cloud | `ANTHROPIC_API_KEY` | Claude Opus, Sonnet, Haiku |
+| **Gemini (Google)** | Cloud | `GEMINI_API_KEY` | Gemini, Veo, Nano Banana |
+| **xAI** | Cloud | `XAI_API_KEY` | Grok |
+| **Ollama** | Local | `OLLAMA_API_URL` | Qwen, Llama 3, Mistral, any GGUF. Separate install, must be running |
+| **HuggingFace** | Cloud | `HF_TOKEN` | Hosted Inference Providers |
+| **HuggingFace local** (`huggingface-local`) | Local | — | Diffusers/Transformers models via the HuggingFace Python pack |
+| **NodeTool** (`nodetool`) | Cloud, managed | — (credits) | Curated models |
+| **FAL** (`fal_ai`) | Cloud | `FAL_API_KEY` | Fast image/video generation |
+| **KIE** (`kie`) | Cloud | `KIE_API_KEY` | Image, video and music models from many vendors |
+| **ElevenLabs** (`elevenlabs`) | Cloud | `ELEVENLABS_API_KEY` | TTS, voice, sound effects |
+| **Replicate** | Cloud | `REPLICATE_API_TOKEN` | Community models |
+| **Comfy Cloud** | Cloud | `COMFY_API_KEY` | No model list; runs a ComfyUI workflow on Comfy's GPUs via `lib.comfy.RunWorkflowOnCloud` |
+| **vLLM** | Local | `VLLM_BASE_URL` (required) | Self-hosted, OpenAI-compatible |
+| **LM Studio** | Local | `LMSTUDIO_API_URL` | Models loaded in LM Studio's local server |
+| **llama.cpp server** (`llama_cpp`) | Local | `LLAMA_CPP_URL` (required) | A `llama-server` you run |
+| **llama.cpp local** (`node_llama_cpp`) | Local | — | GGUF files in-process; runtime from Package Manager → Engines & tools |
+| **MLX** | Local | — | MLX Python pack, Apple Silicon only |
+
+Other registered chat providers (any of these is valid for the CLI's `-p/--provider`):
+`groq`, `mistral`, `deepseek`, `moonshot`, `minimax`, `cerebras`, `alibaba`
+(`DASHSCOPE_API_KEY`, Qwen via Alibaba Cloud Model Studio), `together`,
+`openrouter`, `requesty`, `opper`, `api_route`, `codex`, `claude_agent_sdk`, `lmstudio`,
+`azure_openai` (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, models from
+`AZURE_OPENAI_DEPLOYMENTS`), `bedrock` (`BEDROCK_ACCESS_KEY_ID`,
+`BEDROCK_SECRET_ACCESS_KEY`, `BEDROCK_REGION`).
+
+# From a shell (self-host)
+
+These run in the user's terminal, not in the product. `nodetool models
+providers` shows configured providers, `nodetool models recommended` the curated
+list, and `nodetool models by-provider <provider>` one provider's ids.
+
+```bash
+# Via CLI (encrypted storage)
+nodetool secrets store OPENAI_API_KEY
+nodetool secrets store ANTHROPIC_API_KEY
+nodetool secrets store GEMINI_API_KEY
+nodetool secrets store HF_TOKEN
+nodetool secrets store FAL_API_KEY
+nodetool secrets store REPLICATE_API_TOKEN
+nodetool secrets store COMFY_API_KEY
+
+# Via environment variables
+export OPENAI_API_KEY=sk-...
+export ANTHROPIC_API_KEY=sk-ant-...
+export GEMINI_API_KEY=AI...
+export HF_TOKEN=hf_...
+export FAL_API_KEY=...
+export REPLICATE_API_TOKEN=r8_...
+export COMFY_API_KEY=comfyui-...
+export OLLAMA_API_URL=http://localhost:11434
+```
+
+# Model Selection by Task
+
+The tables name model families as starting points. `find_model` ranks what is
+actually available on this install and returns the exact id.
+
+## whisper.cpp speech recognition
+
+Both providers are local-only and unavailable on the cloud profile.
+`whisper_cpp` runs speech recognition in the backend through the optional
+`@fugood/whisper.node@1.1.3` runtime package. Install **whisper.cpp** from the
+Package Manager, then download a GGML model from Models. It serves
+`nodetool.text.AutomaticSpeechRecognition` and `whisper_cpp.LiveTranscription`.
+Use the absolute model path returned by discovery unchanged as the ASR model id.
+
+Models use the Hugging Face hub cache (see HuggingFace models below for how its
+location resolves). `WHISPER_CPP_MODELS_DIR` adds another directory.
+`WHISPER_CPP_GPU_BACKEND` accepts `auto`, `metal`, `cuda`, `vulkan`, or `cpu`.
+`auto` uses Metal on macOS. On Windows and Linux it tries the CUDA build, then
+the Vulkan build, then the default CPU build. Restart the backend after
+changing the backend setting. Live transcription accepts base64 PCM16
+mono chunks with an optional `content_metadata.sample_rate` and uses Silero
+VAD when installed, otherwise fixed windows.
+
+`whisper_cpp_server` calls a user-run `whisper-server`. Set
+`WHISPER_CPP_SERVER_URL` to its base URL. The model id is `default`, representing
+the model loaded by that server. It requires no native runtime package.
+
+## Language / Chat
+
+| Need | Family | Provider | Notes |
+|------|--------|----------|-------|
+| Best quality | GPT flagship, Claude Opus or Sonnet | OpenAI, Anthropic | Highest capability |
+| Good balance | GPT mini, Gemini Flash | OpenAI, Gemini | Fast + cheap |
+| Local/private | Llama, Qwen | Ollama | No data leaves machine |
+| Lightweight local | Small Llama, Mistral 7B | Ollama | Low memory |
+| Code | Claude Sonnet, GPT flagship | Anthropic, OpenAI | Best for coding |
+
+## Image Generation
+
+| Need | Model | Provider | Notes |
+|------|-------|----------|-------|
+| Best quality | FLUX.2 Dev | HuggingFace, FAL | State-of-art |
+| Fast | FLUX Schnell | HuggingFace | Quick iterations |
+| Versatile | SDXL | HuggingFace | Many LoRAs available |
+| API-based | GPT Image 2, Nano Banana | OpenAI, Gemini/KIE | No local GPU needed |
+
+## Video Generation
+
+| Need | Model | Provider |
+|------|-------|----------|
+| Best quality | Sora 2 Pro | OpenAI (KIE) |
+| Fast | Wan 2.6 | KIE |
+| Image-to-video | Kling 2.6 | KIE |
+| Talking avatar | Kling AI Avatar | KIE |
+
+## Speech & Audio
+
+| Need | Model | Provider |
+|------|-------|----------|
+| TTS (quality) | ElevenLabs | ElevenLabs |
+| TTS (fast/free) | Kokoro | HuggingFace pack, MLX pack, Transformers.js |
+| ASR (accuracy) | Whisper Large V3 | HuggingFace pack, whisper.cpp |
+| ASR (fast) | Whisper Turbo | HuggingFace pack, MLX pack, whisper.cpp |
+
+## Embeddings
+
+| Need | Model | Provider |
+|------|-------|----------|
+| General text | text-embedding-3-small | OpenAI |
+| Best quality | text-embedding-3-large | OpenAI |
+| Local/free | sentence-transformers | HuggingFace |
+
+# Local Model Setup
+
+## Ollama (Easiest)
+
+Ollama is a separate program. NodeTool does not ship or start it, so it must be
+running before its models appear. On macOS and Windows, download it from
+ollama.com. On Linux:
+
+```bash
+# Install
+curl -fsSL https://ollama.com/install.sh | sh
+
+# Pull models
+ollama pull llama3
+ollama pull mistral
+ollama pull qwen2
+
+# Verify
+ollama list
+
+# NodeTool finds Ollama at 127.0.0.1:11434
+# Override: OLLAMA_API_URL in Settings → Integrations → Local Model Servers,
+# or export OLLAMA_API_URL=http://host:11434
+```
+
+`OLLAMA_CONTEXT_LENGTH` sets the context window sent with each request.
+Unset, NodeTool uses the model's own value, capped at 32768.
+
+## Python packs (HuggingFace local, MLX, Wan2GP)
+
+Local Diffusers/Transformers models, MLX models and Wan2GP video run in the
+Python worker, and each needs its pack.
+
+- Desktop app: **Tools → Package Manager → Python nodes**. Installing a pack
+  sets up Python first if it is missing. The HuggingFace pack pulls a PyTorch
+  2.14 build matched to the GPU. MLX is offered only on Apple Silicon Macs.
+  On a Mac, both packs need Apple Silicon and macOS 14 or newer.
+- Without the desktop app: a Python 3.11+ env with
+  `pip install nodetool-core nodetool-huggingface` (plus `nodetool-mlx` on
+  Apple Silicon), then start the server with `NODETOOL_PYTHON` pointing at that
+  env's `python`. `curl -fsSL https://raw.githubusercontent.com/nodetool-ai/nodetool/main/install.sh | bash -s -- -y --pack huggingface`
+  builds that env on Linux and macOS. `--pack` takes `huggingface`, `mlx`, or
+  `wan2gp`. On macOS, always set `NODETOOL_PYTHON` to the env it prints.
+- `NODETOOL_TORCH_DEVICE` (`cuda`, `cuda:1`, `mps`, `cpu`) picks the device.
+- Wan2GP nodes call a Wan2GP server the user runs. `WAN2GP_MCP_URL` (default
+  `http://127.0.0.1:7866/mcp`) or the node's `server_url` points at it.
+
+## HuggingFace models
+
+Download local models in the Model Manager before running the workflow. A node
+that downloads on first use can fail on a gated repo or stall a run.
+
+For gated models:
+1. Accept the license on the model page at huggingface.co
+2. Store `HF_TOKEN` in Settings (or `nodetool secrets store HF_TOKEN`). Model
+   Manager downloads use it
+3. Download the model in the Model Manager
+
+Models go to the Hugging Face hub cache: `HF_HUB_CACHE`, else
+`HUGGINGFACE_HUB_CACHE`, else `$HF_HOME/hub`, else
+`$XDG_CACHE_HOME/huggingface/hub`, else `~/.cache/huggingface/hub`. The
+server and the Python worker (through `huggingface_hub`) apply this one order.
+The desktop app keeps `~/.cache/huggingface` when `XDG_CACHE_HOME` is set (as
+in Flatpak) but that cache already holds models.
+
+## llama.cpp
+
+Two providers. `node_llama_cpp` loads GGUF files in-process after the runtime
+is installed from Package Manager → Engines & tools. `llama_cpp` calls a
+`llama-server` the user runs, at `LLAMA_CPP_URL`. Best for CPU inference and
+quantized models.
+
+## whisper.cpp and Transformers.js
+
+whisper.cpp is speech recognition only (see above). Transformers.js runs small
+ONNX models in-process after its runtime is installed from Package Manager →
+Software. Its cache is `TRANSFORMERS_JS_CACHE_DIR`.
+
+## MLX (Apple Silicon only)
+
+Needs the MLX pack and macOS 14 or newer. Lower memory usage than standard
+PyTorch.
+
+# Local Inference Performance
+
+| Framework | Throughput | Memory | Hardware |
+|-----------|-----------|--------|----------|
+| **llama.cpp** | Medium | Excellent | CPU, GPU |
+| **MLX** | Good | Excellent | Apple Silicon |
+| **Transformers** | Medium | Good | Any |
+
+# Provider-Agnostic Nodes
+
+These nodes work with any provider — just select the model:
+
+| Node | Purpose |
+|------|---------|
+| `nodetool.agents.Agent` | Any LLM for chat/reasoning |
+| `nodetool.image.TextToImage` | Any image generation model |
+| `nodetool.image.ImageToImage` | Any image transformation model |
+| `nodetool.video.TextToVideo` | Any video generation model |
+| `nodetool.video.ImageToVideo` | Any image-to-video model |
+| `nodetool.audio.TextToSpeech` | Any TTS model |
+| `nodetool.text.AutomaticSpeechRecognition` | Any ASR model |
+
+# Provider Capabilities
+
+| Capability | OpenAI | Anthropic | Google | Ollama | HF |
+|-----------|--------|-----------|--------|--------|-----|
+| Chat/Text | yes | yes | yes | yes | yes |
+| Vision | yes | yes | yes | some | yes |
+| Image Gen | yes (GPT-Image) | no | no | no | yes |
+| Video Gen | no | no | yes (Veo) | no | some |
+| TTS | yes | no | no | no | yes |
+| ASR | yes (Whisper) | no | no | no | yes |
+| Embeddings | yes | no | yes | yes | yes |
+| Tool Calling | yes | yes | yes | some | no |
+
+# Common Pitfalls
+
+- **Wrong key env var name**: Each provider has a specific name (see table above)
+- **Ollama not running**: NodeTool does not start it. Start the Ollama app or `ollama serve`
+- **Gated HF models**: Must accept terms on huggingface.co first
+- **GPU memory**: Large models need 8-24GB VRAM; use quantized versions
+- **PyTorch on the CPU despite a GPU**: the desktop app installs CUDA 12.8 for Turing (RTX 20xx) and newer (driver 570+), CUDA 12.6 for Maxwell to Volta, and CPU for Kepler, pre-RDNA 2 AMD and AMD on Windows. It also falls back to CPU, with a warning, when no GPU build resolves. A manual `pip install torch` on Linux gets CUDA 13.0, which needs driver 580+ and Turing or newer, as does the Docker worker image
+- **No local provider in the model menu**: the Python pack is missing, or `NODETOOL_PYTHON_ON_DEMAND=true` and no workflow has started the worker yet
+- **Rate limits**: Cloud providers have rate limits; implement retries or use local
+- **Model ID mismatch**: Take ids from `find_model` or `list_provider_models`, never from memory. From a shell, `nodetool models by-provider <provider>` lists them
+- **Key pasted into chat**: Use `request_secret`. A key in a chat message is stored in the thread
+
+# Repository only: custom provider
+
+This is TypeScript for contributors working in a NodeTool checkout. The
+in-product agent cannot add a provider. Providers extend `BaseProvider` from `@nodetool-ai/runtime` (not `@nodetool-ai/core`).
+Both `generateMessage` and `generateMessages` take a single **args object**.
+
+```typescript
+import {
+  BaseProvider,
+  type ProviderId,
+  type Message,
+  type ProviderStreamItem,
+  type ProviderTool,
+  type LanguageModel,
+} from "@nodetool-ai/runtime";
+
+export class MyProvider extends BaseProvider {
+  private apiKey: string;
+
+  constructor(kwargs: Record<string, unknown> = {}) {
+    super("my_provider" as ProviderId);
+    this.apiKey = String(kwargs["MY_API_KEY"] ?? process.env.MY_API_KEY ?? "");
+  }
+
+  static override requiredSecrets(): string[] {
+    return ["MY_API_KEY"];
+  }
+
+  // Non-streaming: return a single assistant Message.
+  async generateMessage(args: {
+    messages: Message[];
+    model: string;
+    tools?: ProviderTool[];
+  }): Promise<Message> {
+    // Call your API with args.messages / args.model …
+    return { role: "assistant", content: "response text" };
+  }
+
+  // Streaming: yield ProviderStreamItem chunks.
+  async *generateMessages(args: {
+    messages: Message[];
+    model: string;
+    tools?: ProviderTool[];
+  }): AsyncGenerator<ProviderStreamItem> {
+    yield { type: "chunk", content: "response text", done: false };
+  }
+
+  override async getAvailableLanguageModels(): Promise<LanguageModel[]> {
+    return [{ id: "my-model", name: "My Model", provider: "my_provider" }];
+  }
+}
+```
+
+Register it with `registerProvider("my_provider", MyProvider)` from
+`@nodetool-ai/runtime`.