waypoi 0.0.0

package/docs/mcp-service.md

@@ -0,0 +1,178 @@
+# MCP Service (`/mcp`)
+
+Waypoi provides a built-in MCP server at:
+
+- `POST /mcp` (Streamable HTTP transport)
+- localhost only (`localhost`, `127.0.0.1`, `::1`)
+
+Policy authority: [`docs/mcp-guidelines.md`](./mcp-guidelines.md)
+
+## Client flow
+
+1. `initialize`
+2. `notifications/initialized`
+3. `tools/list`
+4. `tools/call`
+
+## Tool: `generate_image`
+
+Generate image(s) from text using Waypoi's diffusion routing.
+When `image_path` or `image_url` is provided, the tool performs image-to-image editing.
+
+Governance note: this tool follows the file-first policy from [`docs/mcp-guidelines.md`](./mcp-guidelines.md).
+
+### Server environment guards
+
+`generate_image` file-output path can be overridden with server env vars:
+
+- `WAYPOI_MCP_OUTPUT_ROOT` (optional, absolute path):
+  - sets the output base directory; defaults to `~/.config/waypoi` when unset
+- `WAYPOI_MCP_OUTPUT_SUBDIR` (optional, relative path):
+  - narrows output to `<WAYPOI_MCP_OUTPUT_ROOT>/<subdir>` instead of `<root>/generated-images`
+  - example: `work`
+- `WAYPOI_MCP_STRICT_OUTPUT_ROOT` (optional, `true|false`, default `false`):
+  - when `true`, `WAYPOI_MCP_OUTPUT_ROOT` is required and must be absolute
+  - invalid/missing strict config returns typed `invalid_request` errors
+
+Example for pinning outputs to a specific project:
+
+```bash
+export WAYPOI_MCP_OUTPUT_ROOT=/path/to/project
+export WAYPOI_MCP_OUTPUT_SUBDIR=work
+export WAYPOI_MCP_STRICT_OUTPUT_ROOT=true
+```
+
+### Input fields
+
+- `prompt` (required, string)
+- `model` (optional, string)
+- `image_path` (optional, string; local file path for image-to-image editing)
+- `image_url` (optional, string; supports `http(s)` or `data:` URL for image-to-image editing)
+- `n` (optional, integer `1..4`)
+- `size` (optional, string)
+- `quality` (optional, string)
+- `style` (optional, string)
+- `response_format` (optional, `"url"` or `"b64_json"`)
+- `include_data` (optional, boolean):
+  - include `url`/`b64_json` in detailed structured output
+  - default is `false`
+
+### File-output behavior
+
+`generate_image` always writes image bytes to disk and returns metadata:
+
+- `file_path`
+- `file_paths` when multiple images are generated
+- `mime_type`
+- `bytes`
+
+Default output directory: `~/.config/waypoi/generated-images` (or `$WAYPOI_DIR/generated-images`).
+Override with `WAYPOI_MCP_OUTPUT_ROOT` and optionally `WAYPOI_MCP_OUTPUT_SUBDIR`.
+
+Returned `file_path` values are relative to `WAYPOI_MCP_OUTPUT_ROOT` (or `~/.config/waypoi`).
+
+Implementation note: `generate_image` forces upstream `response_format` to `"b64_json"` to ensure bytes are always available for writing, even if caller passes `"url"`.
+
+### Validation rules
+
+- `image_path` and `image_url` are mutually exclusive.
+- If `WAYPOI_MCP_OUTPUT_SUBDIR` is set, it must be a relative path.
+- If `WAYPOI_MCP_STRICT_OUTPUT_ROOT=true`, `WAYPOI_MCP_OUTPUT_ROOT` must be set and absolute.
+
+### Response notes
+
+- Success: `ok: true` with `summary`, `file_path`/`file_paths`, and detailed `artifacts[]`.
+- Error: `ok: false` with typed `error` (`invalid_request`, `no_diffusion_model`, `upstream_error`, etc).
+
+## Tool: `understand_image`
+
+Analyze an image using a vision-capable text model and return structured text.
+
+### Input fields
+
+- `image_path` (optional, string; local file path)
+- `image_url` (optional, string; supports `http(s)` or `data:` URL)
+- `instruction` (optional, string; default: general OCR/object/scene/detail analysis)
+- `model` (optional, string; auto-selects best vision-capable text-output model when omitted)
+- `max_tokens` (optional, integer `1..4096`)
+- `temperature` (optional, number `0..2`)
+
+Validation:
+
+- Exactly one image source is required: `image_path` XOR `image_url`.
+
+### Response shape
+
+Success:
+
+- `ok`
+- `summary`
+- `model`
+- `text`
+- `result`:
+  - `answer`
+  - `ocr_text`
+  - `objects`
+  - `scene`
+  - `notable_details`
+  - `safety_notes`
+- `image_geometry` (optional for local image paths):
+  - `original_width`
+  - `original_height`
+  - `uploaded_width`
+  - `uploaded_height`
+  - `scale_x`
+  - `scale_y`
+  - `resized`
+- `usage` (`prompt_tokens`, `completion_tokens`, `total_tokens`)
+
+For local `image_path` inputs, Waypoi may resize the uploaded image before sending it upstream. When that happens, it prepends a system instruction telling the model to report any coordinates in original-image pixels and includes `image_geometry` in the success payload for debugging and downstream correction.
+
+Error:
+
+- `ok: false`
+- typed `error` (`invalid_request`, `no_vision_model`, `upstream_error`, ...)
+
+## Tool: `generate_video`
+
+Generate a video from a text prompt or image using Alibaba Cloud ModelStudio (Wan models).
+Supports text-to-video and image-to-video generation. Videos are generated asynchronously
+and may take 1-5 minutes.
+
+Governance note: video URLs are returned directly (not written to disk) because videos are
+streaming media files typically consumed via URL rather than embedded.
+
+### Input fields
+
+- `prompt` (required, string): Detailed description of the desired video content, style, camera movement, and lighting.
+- `model` (optional, string): Model name (e.g., `wan2.6-i2v-flash`, `wan2.6-t2v`). Auto-selected when omitted.
+- `image_url` (optional, string): URL or data URL of the first-frame image for image-to-video generation.
+- `audio_url` (optional, string): URL of an audio file to sync with the video (wan2.6/wan2.5 models only).
+- `duration` (optional, integer `2..15`): Video duration in seconds. Model-dependent defaults apply.
+- `resolution` (optional, `"480P"`, `"720P"`, or `"1080P"`): Output resolution tier.
+- `negative_prompt` (optional, string): Description of content to exclude.
+- `seed` (optional, integer `0..2147483647`): Random seed for reproducibility.
+- `watermark` (optional, boolean): Add "AI Generated" watermark (default: false).
+- `prompt_extend` (optional, boolean): Enable intelligent prompt rewriting (default: true).
+
+### Response shape
+
+Success:
+
+- `ok: true`
+- `summary`: "Generated 1 video."
+- `model`: upstream model name used
+- `url`: URL to the generated MP4 video (valid for 24 hours)
+- `revised_prompt` (optional): optimized prompt if prompt_extend was enabled
+- `video_count`, `duration`, `resolution` (optional): usage metadata
+
+Error:
+
+- `ok: false`
+- typed `error` (`invalid_request`, `no_video_model`, `no_video_output`, `upstream_error`, ...)
+
+### Response notes
+
+- Video URLs expire after 24 hours. Download and save videos promptly to permanent storage.
+- The `generate_video` tool has a 5-minute timeout to accommodate async generation.
+- Output video format: MP4 (H.264 encoding), 30 fps.

package/docs/opencode.md

@@ -0,0 +1,86 @@
+# Using Waypoi with Opencode
+
+Waypoi is a local OpenAI-compatible gateway. Opencode should connect to Waypoi as an external client.
+
+## 1) Start Waypoi
+
+```bash
+waypoi provider import -f .env
+npm run start
+```
+
+Default base URL:
+
+- `http://localhost:9469/v1`
+
+## 2) Point Opencode to Waypoi
+
+Configure Opencode with:
+
+- Base URL: `http://localhost:9469/v1`
+- API key: `local-dev` (or your configured auth token)
+
+If Waypoi auth is enabled, send the matching bearer token.
+
+## 3) Validate model discovery
+
+```bash
+curl http://localhost:9469/v1/models
+```
+
+Waypoi returns an OpenAI-style model list from healthy endpoints and smart pool aliases.
+Use `smart` as the default model for free-tier routing with automatic failover.
+
+## 4) Optional: Responses API
+
+Waypoi exposes:
+
+- `POST /v1/responses`
+
+Use this only when your client supports Responses API style requests. Otherwise use chat completions.
+
+## 5) Basic request check
+
+```bash
+curl http://localhost:9469/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer local-dev" \
+  -d '{
+    "model": "smart",
+    "messages": [{"role": "user", "content": "hello"}]
+  }'
+```
+
+## 6) Benchmark the same real-world flows
+
+Use the built-in showcase examples to replay the same kinds of interactions you expect from Opencode:
+
+```bash
+# See available examples
+waypoi bench --list-examples
+
+# Plain chat
+waypoi bench --example showcase-chat-welcome
+
+# Responses API compatibility
+waypoi bench --example showcase-responses-basic
+
+# Tool-calling live show
+waypoi bench --example showcase-agent-tool-call
+```
+
+The Benchmark UI shows:
+
+- the exact scenario input
+- the wire request Waypoi sent
+- tool calls and tool results
+- the final model response
+- the final verdict
+
+This is the fastest way to demonstrate that an Opencode-style workflow works end to end.
+
+## Troubleshooting
+
+- Empty model list: verify endpoint health with `waypoi status`.
+- 401/403: check auth mode and token.
+- Slow/failover behavior: inspect `waypoi stats` and dashboard latency panels.

package/docs/providers.md

@@ -0,0 +1,79 @@
+# Providers and Protocol Adapters
+
+Waypoi uses a provider catalog with protocol adapters.  
+External clients still use OpenAI-compatible `/v1/*` endpoints; adapters translate to provider-native protocols.
+
+## Onboarding a non-OpenAI protocol provider
+
+1. Create a provider YAML with:
+   - `endpoint.protocol`
+   - `endpoint.baseUrl`
+   - protocol-specific config (for `inference_v2`: `endpoint.router`)
+2. Import with:
+   - `waypoi providers import --registry <registry.yaml> --env-file .env`
+3. Rebuild pools:
+   - `waypoi providers import ...` (default auto rebuild) or `POST /admin/pools/rebuild`
+4. Verify:
+   - `waypoi providers`
+   - `waypoi providers show <id>`
+   - `waypoi models <id>`
+5. Optional TLS policy:
+   - `waypoi providers update <id> --insecure-tls|--strict-tls`
+   - `waypoi providers update <id> --auto-insecure-domain <suffix...>`
+
+## Provider model CRUD
+
+Use provider-first model management:
+
+- `waypoi models add <providerId> --model-id <id> --upstream <name> --base-url <url>`
+- `waypoi models update <providerId> <modelRef> ...`
+- `waypoi models update <providerId> <modelRef> --clear-insecure-tls`
+- `waypoi models rm <providerId> <modelRef>`
+- `waypoi models enable <providerId>/<modelRef>`
+- `waypoi models disable <providerId>/<modelRef>`
+- `waypoi models set-key <providerId>/<modelRef> --api-key <key>`
+
+Legacy command forms (`waypoi provider ...`, `waypoi provider model ...`) are still supported through rewrite shims with deprecation warnings.
+
+Legacy endpoint write commands are blocked in v0.5.0; use migration + `waypoi models ...` commands.
+
+## Inference V2 (ray/kserve-style) example
+
+See: `/Users/zziang/Documents/projects/vibeCoding/Agents/waypoi/examples/providers/inference-v2-ray.yaml`
+
+Key fields:
+
+- `endpoint.protocol: inference_v2`
+- `endpoint.router: <router_name>`
+- `endpoint.responseTextPaths` (optional response extraction path list)
+
+## Notes
+
+- `inference_v2` v1 supports chat/vision sync (`stream=false`) only.
+- Streaming requests are rejected for this protocol unless another pool candidate supports streaming.
+- Unknown protocols are imported but marked non-routable.
+- Pool alias surface is now a single `smart` alias; legacy `smart-*` aliases are rejected.
+- TLS inheritance:
+  - Effective TLS mode is `model.insecureTls ?? provider.insecureTls ?? false`.
+  - Models added without `--insecure-tls` inherit provider TLS mode.
+- Allowlisted auto-insecure fallback:
+  - On TLS verify failures, Waypoi retries insecure TLS only when hostname matches provider `autoInsecureTlsDomains`.
+  - If retry succeeds, Waypoi persists `model.insecureTls=true` for that model.
+
+## PCAI endpoint migration runbook (`*.ai-application.stjude.org`)
+
+This migration copies endpoint-managed models into provider `pcai`, then disables the source endpoints.
+
+1. Pre-check:
+   - `waypoi ls`
+2. Run migration:
+   - `waypoi providers migrate-endpoints --provider pcai --match-domain ai-application.stjude.org --protocol openai`
+3. Verify:
+   - `waypoi providers show pcai`
+   - `waypoi models pcai`
+   - `waypoi providers pools`
+   - `waypoi ls` (legacy endpoints should show `disabled=yes`)
+4. Rollback (single model path):
+   - Re-enable the endpoint in `config.yaml` (`disabled: false`) or via admin endpoint patch.
+   - Set the corresponding `pcai` provider model `enabled: false` in `$WAYPOI_DIR/providers.json`.
+   - Rebuild pools: `waypoi providers pools` (or `POST /admin/pools/rebuild`).

package/examples/benchmark.config.yaml

@@ -0,0 +1,28 @@
+version: 1
+
+defaults:
+  requestTimeoutMs: 120000
+  toolTimeoutMs: 15000
+  maxIterations: 6
+  temperature: 0
+  top_p: 1
+  max_tokens: 512
+  presence_penalty: 0
+  frequency_penalty: 0
+
+profiles:
+  local:
+    warmupRuns: 1
+    measuredRuns: 3
+    minScenarioPassRate: 1.0
+  ci:
+    warmupRuns: 2
+    measuredRuns: 5
+    minScenarioPassRate: 1.0
+
+gates:
+  hard:
+    smokeMinSuccessRate: 1.0
+  soft:
+    maxP95RegressionPct: 20
+    maxThroughputDropPct: 20

package/examples/providers/alibaba-dashscope.yaml

@@ -0,0 +1,88 @@
+$id: alibaba-dashscope
+name: Alibaba Cloud Model Studio (DashScope Native)
+description: Wan series models for image and video generation via DashScope native API
+docs: https://www.alibabacloud.com/help/en/model-studio/wan-image-generation-api-reference
+endpoint:
+  baseUrl: https://dashscope-intl.aliyuncs.com
+  protocol: dashscope
+auth:
+  type: bearer
+  keyParam: Authorization
+env: DASHSCOPE_API_KEY
+models:
+  - id: wan2.6-image
+    upstream: wan2.6-image
+    free: false
+    modalities:
+      - text-to-image
+    capabilities:
+      streaming: false
+    benchmark:
+      livebench: 60.0
+  - id: wan2.6-i2v-flash
+    upstream: wan2.6-i2v-flash
+    free: false
+    modalities:
+      - text-to-video
+      - image-to-video
+    capabilities:
+      streaming: false
+    benchmark:
+      livebench: 58.0
+  - id: wan2.6-i2v
+    upstream: wan2.6-i2v
+    free: false
+    modalities:
+      - text-to-video
+      - image-to-video
+    capabilities:
+      streaming: false
+    benchmark:
+      livebench: 57.0
+  - id: wan2.6-t2v
+    upstream: wan2.6-t2v
+    free: false
+    modalities:
+      - text-to-video
+    capabilities:
+      streaming: false
+    benchmark:
+      livebench: 56.0
+  - id: wan2.5-i2v-preview
+    upstream: wan2.5-i2v-preview
+    free: false
+    modalities:
+      - text-to-video
+      - image-to-video
+    capabilities:
+      streaming: false
+    benchmark:
+      livebench: 55.0
+  - id: wan2.5-t2v-preview
+    upstream: wan2.5-t2v-preview
+    free: false
+    modalities:
+      - text-to-video
+    capabilities:
+      streaming: false
+    benchmark:
+      livebench: 54.0
+  - id: wan2.2-i2v-plus
+    upstream: wan2.2-i2v-plus
+    free: false
+    modalities:
+      - text-to-video
+      - image-to-video
+    capabilities:
+      streaming: false
+    benchmark:
+      livebench: 50.0
+  - id: wan2.2-t2v-plus
+    upstream: wan2.2-t2v-plus
+    free: false
+    modalities:
+      - text-to-video
+    capabilities:
+      streaming: false
+    benchmark:
+      livebench: 49.0

package/examples/providers/alibaba-llm.yaml

@@ -0,0 +1,64 @@
+$id: alibaba-llm
+name: Alibaba Cloud Model Studio (OpenAI Compatible)
+description: Qwen series models via OpenAI-compatible interface
+docs: https://www.alibabacloud.com/help/en/model-studio/compatibility-of-openai-with-dashscope
+endpoint:
+  baseUrl: https://dashscope-intl.aliyuncs.com/compatible-mode/v1
+  protocol: openai
+auth:
+  type: bearer
+  keyParam: Authorization
+env: DASHSCOPE_API_KEY
+models:
+  - id: qwen3-max
+    upstream: qwen3-max
+    free: false
+    modalities:
+      - text-to-text
+    capabilities:
+      streaming: true
+      tools: true
+    benchmark:
+      livebench: 72.0
+  - id: qwen-plus
+    upstream: qwen-plus
+    free: false
+    modalities:
+      - text-to-text
+    capabilities:
+      streaming: true
+      tools: true
+    benchmark:
+      livebench: 68.0
+  - id: qwen-flash
+    upstream: qwen-flash
+    free: false
+    modalities:
+      - text-to-text
+    capabilities:
+      streaming: true
+      tools: true
+    benchmark:
+      livebench: 62.0
+  - id: qwen3.5-plus
+    upstream: qwen3.5-plus
+    free: false
+    modalities:
+      - text-to-text
+      - image-to-text
+    capabilities:
+      streaming: true
+      tools: true
+      vision: true
+    benchmark:
+      livebench: 70.0
+  - id: qwen3-coder-plus
+    upstream: qwen3-coder-plus
+    free: false
+    modalities:
+      - text-to-text
+    capabilities:
+      streaming: true
+      tools: true
+    benchmark:
+      livebench: 65.0

package/examples/providers/alibaba-registry.yaml

@@ -0,0 +1,7 @@
+providers:
+  - id: alibaba-llm
+    name: Alibaba Cloud Model Studio (OpenAI Compatible)
+    file: alibaba-llm.yaml
+  - id: alibaba-dashscope
+    name: Alibaba Cloud Model Studio (DashScope Native)
+    file: alibaba-dashscope.yaml

package/examples/providers/inference-v2-ray.yaml

@@ -0,0 +1,29 @@
+$id: ray-gateway
+name: Ray Gateway Router
+description: Example provider using /v2/models/{router}/infer protocol
+docs: https://ray.io/
+endpoint:
+  baseUrl: https://svltgpt01a.stjude.org
+  protocol: inference_v2
+  router: ray_gateway_router
+  responseTextPaths:
+    - outputs.0.outputs.text
+    - outputs.0.text
+auth:
+  type: bearer
+  keyParam: Authorization
+env: RAY_GATEWAY_API_KEY
+models:
+  - id: qwen3-vl-30b
+    upstream: qwen3-VL-30B-A3B-Thinking-vllm
+    free: false
+    modalities:
+      - text-to-text
+      - image-to-text
+    capabilities:
+      streaming: false
+      tools: false
+      vision: true
+    benchmark:
+      livebench: 52.0
+

package/examples/scenarios/custom.jsonl

@@ -0,0 +1,5 @@
+{"id":"jsonl-chat-1","mode":"chat","prompt":"Say WAYPOINT_JSONL_OK","assertions":{"contains":"WAYPOINT_JSONL_OK","statusCode":200}}
+{"id":"jsonl-agent-1","mode":"agent","prompt":"Use a tool if available then answer.","assertions":{"statusCode":200}}
+{"id":"jsonl-emb-1","mode":"embeddings","input":"waypoint jsonl embedding","assertions":{"minItems":1,"minVectorLength":1,"statusCode":200}}
+{"id":"jsonl-img-1","mode":"image_generation","prompt":"A simple blue square","assertions":{"minImages":1,"statusCode":200}}
+{"id":"jsonl-tts-1","mode":"audio_speech","inputText":"Waypoint jsonl speech","voice":"alloy","assertions":{"minBytes":1,"statusCode":200}}

package/examples/scenarios/custom.yaml

@@ -0,0 +1,40 @@
+scenarios:
+  - id: custom-chat-short
+    mode: chat
+    prompt: "Reply with WAYPOI_CUSTOM_OK"
+    assertions:
+      contains: "WAYPOI_CUSTOM_OK"
+      statusCode: 200
+
+  - id: custom-agent-tools
+    mode: agent
+    prompt: "If tools are available, call one and summarize with prefix WAYPOI_AGENT_DONE:"
+    maxIterations: 5
+    assertions:
+      contains: "WAYPOI_AGENT_DONE"
+      statusCode: 200
+
+  - id: custom-embeddings
+    mode: embeddings
+    input:
+      - "waypoi"
+      - "benchmark"
+    assertions:
+      minItems: 2
+      minVectorLength: 1
+      statusCode: 200
+
+  - id: custom-image
+    mode: image_generation
+    prompt: "A clean neon gateway symbol"
+    assertions:
+      minImages: 1
+      statusCode: 200
+
+  - id: custom-tts
+    mode: audio_speech
+    inputText: "Waypoi benchmark audio"
+    voice: "alloy"
+    assertions:
+      minBytes: 1
+      statusCode: 200

package/package.json

@@ -0,0 +1,66 @@
+{
+  "name": "waypoi",
+  "version": "0.0.0",
+  "description": "Local OpenAI-compatible AI gateway with intelligent routing, failover, MCP tools, and a built-in playground UI",
+  "license": "MIT",
+  "main": "dist/src/index.js",
+  "bin": {
+    "waypoi": "dist/cli/index.js",
+    "waypoi-dev": "dist/cli/index.js"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/ziangziangziang/waypoi.git"
+  },
+  "homepage": "https://github.com/ziangziangziang/waypoi#readme",
+  "bugs": {
+    "url": "https://github.com/ziangziangziang/waypoi/issues"
+  },
+  "keywords": [
+    "ai",
+    "llm",
+    "proxy",
+    "openai",
+    "mcp",
+    "local-ai",
+    "gateway",
+    "playground"
+  ],
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "predev": "node scripts/version-from-git.js",
+    "dev": "concurrently -n server,ui -c blue,green \"WAYPOI_DEV=1 tsx watch src/index.ts\" \"VITE_BACKEND_PORT=9470 npm run dev --prefix ui\"",
+    "predev:server": "node scripts/version-from-git.js",
+    "dev:server": "WAYPOI_DEV=1 tsx watch src/index.ts",
+    "predev:all": "node scripts/version-from-git.js",
+    "dev:all": "concurrently -n server,ui -c blue,green \"WAYPOI_DEV=1 tsx watch src/index.ts\" \"VITE_BACKEND_PORT=9470 npm run dev --prefix ui\"",
+    "prebuild": "node scripts/version-from-git.js",
+    "build": "tsc -p tsconfig.json && cd ui && npm run build",
+    "build:ui": "cd ui && npm run build",
+    "build:all": "npm run build",
+    "start": "node dist/src/index.js",
+    "cli": "tsx cli/index.ts",
+    "manual-test": "bash scripts/manual-test.sh",
+    "test": "npx tsx --test tests/*.test.ts",
+    "pretypecheck": "node scripts/version-from-git.js",
+    "typecheck": "tsc --noEmit"
+  },
+  "dependencies": {
+    "@fastify/static": "^9.0.0",
+    "@modelcontextprotocol/sdk": "^1.27.1",
+    "commander": "^12.1.0",
+    "fastify": "^5.8.4",
+    "sharp": "^0.33.5",
+    "undici": "^6.19.8",
+    "yaml": "^2.4.5",
+    "zod": "^4.3.6"
+  },
+  "devDependencies": {
+    "@types/node": "^20.14.10",
+    "concurrently": "^9.1.0",
+    "tsx": "^4.16.2",
+    "typescript": "^5.5.4"
+  }
+}