@tangle-network/starter-foundry 0.5.3 → 0.7.0

package/registry/layers/framework/triton-server/files/AGENTS.md

@@ -0,0 +1,71 @@
+# {{serviceName}} — NVIDIA Triton Inference Server
+
+## What this scaffold gives you
+
+| File | Purpose |
+|---|---|
+| `docker-compose.yml` | Triton container with GPU device reservation, model_repository mount, ports {{port}}/{{grpcPort}}/{{metricsPort}} |
+| `model_repository/identity/config.pbtxt` | Built-in echo model — passes smoke-test without any real weights |
+| `model_repository/identity/1/.gitkeep` | Integer version-dir placeholder (Triton ignores non-integer dirs like `v1/`) |
+| `src/client.py` | HTTP (KServe v2) client — infers against the identity model out of the box |
+| `src/__init__.py` | Package init so `python -m src.client` resolves without a separate `__main__.py` |
+| `requirements.txt` | Runtime pip deps: `tritonclient[http]`, `numpy` |
+| `pyproject.toml` | Python project metadata, deps, pyright (basic), ruff |
+| `Makefile` | Workflow shortcuts: `install`, `install-dev`, `typecheck`, `up`, `health`, `client` |
+| `docs/triton.md` | Extended reference: ONNX/TRT config, dynamic batching, ensemble setup |
+
+## Quick start
+
+```sh
+# 1. Install Python deps
+make install        # pip install -r requirements.txt
+make install-dev    # pip install -e '.[dev]'  (pyright + ruff)
+
+# 2. Start Triton (pulls ~15 GB on first run)
+make up             # docker compose up -d
+
+# 3. Wait for ready (Triton takes ~60 s on cold start)
+make health         # curl -f http://localhost:{{port}}/v2/health/ready
+
+# 4. Run inference smoke-test against the identity model
+make client         # python -m src.client  →  output: [1.0, 2.0, 3.0, 4.0]
+
+# 5. Typecheck
+make typecheck      # pyright src/  — zero errors out of the box
+```
+
+## Add a real model
+
+1. Drop weights under `model_repository/<name>/<version>/` — version **must** be an integer:
+   ```
+   model_repository/
+   └── my_model/
+       ├── config.pbtxt
+       └── 1/
+           └── model.onnx
+   ```
+2. Write `config.pbtxt` — use `backend` **or** `platform` (never both):
+   - ONNX: `backend: "onnxruntime"`
+   - TensorRT: `platform: "tensorrt_plan"`
+   - Python custom: `backend: "python"`
+3. Triton polls `model_repository/` every 30 s. Force an immediate reload:
+   ```sh
+   curl -X POST http://localhost:{{port}}/v2/repository/models/my_model/load
+   ```
+4. Edit `src/client.py` — update `MODEL_NAME`, input/output tensor names, and shape to match your model.
+
+## Key extension points
+
+- **New model** — add `model_repository/<name>/config.pbtxt` + integer version dir with weights
+- **Ensemble** — add `ensemble_scheduling` block in `config.pbtxt` to chain existing models
+- **Client code** — edit `src/client.py`; run `make client` to iterate
+- **Pin the image** — replace `nvcr.io/nvidia/tritonserver:latest` with a specific tag (e.g. `24.10-py3`) in `docker-compose.yml` to avoid surprise breaks on TRT engine rebuilds
+
+## Critical gotchas
+
+- **Integer version dirs only.** `1/`, `2/` work; `v1/`, `latest/` are silently skipped — no error, no warning.
+- **`platform` and `backend` are mutually exclusive.** Mixing them causes an opaque model-load failure.
+- **Dynamic dims** — use `-1` in `dims: [ -1 ]` for variable-size axes. Wrong dims → `inference failed` with no usable stack trace.
+- **TRT engines are GPU-specific.** A `.plan` built on A100 will not load on H100 — rebuild per deploy target.
+- **KServe v2 protocol** — endpoints are `/v2/models/<name>/infer`, not the legacy `/api/infer`.
+- **Image is ~15 GB.** First `docker compose up` takes 10–20 min. Pin the tag so you don't re-pull unexpectedly.

package/registry/layers/framework/triton-server/files/Makefile

@@ -0,0 +1,25 @@
+.PHONY: install install-dev typecheck lint client up logs health
+
+install:
+	pip install -r requirements.txt
+
+install-dev:
+	pip install -e ".[dev]"
+
+typecheck:
+	pyright src/
+
+lint:
+	ruff check src/
+
+client:
+	python -m src.client
+
+up:
+	docker compose up -d
+
+logs:
+	docker compose logs -f triton
+
+health:
+	curl -f http://localhost:8000/v2/health/ready

package/registry/layers/framework/triton-server/files/__init__.py

@@ -0,0 +1 @@
+"""Triton client package. See src/client.py."""

package/registry/layers/framework/triton-server/files/pyproject.toml

@@ -0,0 +1,33 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "starter-triton"
+version = "0.1.0"
+requires-python = ">=3.11"
+dependencies = [
+    "tritonclient[http]>=2.48.0",
+    "numpy>=1.26.0,<2.0.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pyright>=1.1.380",
+    "ruff>=0.6.0",
+]
+
+[tool.setuptools.packages.find]
+include = ["src*"]
+
+[tool.pyright]
+pythonVersion = "3.11"
+typeCheckingMode = "basic"
+include = ["src"]
+reportMissingTypeStubs = "none"
+
+[tool.ruff]
+target-version = "py311"
+
+[tool.ruff.lint]
+select = ["E", "F", "I"]

package/registry/layers/framework/triton-server/manifest.json

@@ -0,0 +1,70 @@
+{
+  "id": "triton-server",
+  "description": "Python project scaffold for NVIDIA Triton Inference Server — src/ package, pyproject.toml with pyright/ruff config, and Makefile targets for install / typecheck / up / client / health.",
+  "appliesTo": ["triton-server"],
+  "files": [
+    { "source": "files/__init__.py",    "target": "src/__init__.py" },
+    { "source": "files/pyproject.toml", "target": "pyproject.toml" },
+    { "source": "files/Makefile",       "target": "Makefile" },
+    { "source": "files/AGENTS.md",      "target": "AGENTS.md" }
+  ],
+  "buildHints": {
+    "whenToUse": "This layer supplies the Python project wrapper for the Triton family — pyproject.toml metadata, a type-checker baseline, and a Makefile so developers can run common tasks without looking up flags.",
+    "firstSteps": [
+      "make install          # pip install -r requirements.txt (tritonclient + numpy)",
+      "make install-dev      # pip install -e '.[dev]' (pyright + ruff)",
+      "make typecheck        # pyright src/ — should be clean out of the box",
+      "make up               # docker compose up -d  (pulls ~15 GB on first run)",
+      "make health           # curl /v2/health/ready — wait for HTTP 200",
+      "make client           # python -m src.client — infer against the identity echo model"
+    ],
+    "gotchas": [
+      "pyproject.toml sets typeCheckingMode=basic; tighten to strict only after adding py.typed and full annotations throughout src/.",
+      "The Makefile uses hard tabs — editors that silently convert to spaces will break make. Most editors detect Makefile type and preserve tabs.",
+      "`make install-dev` requires pyproject.toml to be present (supplied by this layer); running it before layer application will fail.",
+      "pyright is a dev-only dep; the runtime Triton container has no Python typechecker dependency — requirements.txt stays lean.",
+      "model_repository version directories MUST be integers (`1`, `2`, ...). Triton silently skips non-integer dirs like `v1` or `latest`.",
+      "config.pbtxt `platform` and `backend` are mutually exclusive per model — use `backend: \"onnxruntime\"` for ONNX, `platform: \"tensorrt_plan\"` for TRT plans.",
+      "TRT engines (`.plan`) are built for a specific GPU + CUDA + TRT version. An engine built on A100 will not run on H100. Rebuild per deploy target."
+    ],
+    "architectureNotes": [
+      "src/__init__.py makes src/ a proper package so `python -m src.client` resolves without a separate __main__.py.",
+      "pyproject.toml is the single source of truth for Python version, project name, and dev-tool config — requirements.txt is kept as a flat pip-installable subset for Docker and CI.",
+      "The Makefile `health` target hits /v2/health/ready (KServe v2 spec) — not /api/infer, which belongs to the Triton v1 protocol."
+    ],
+    "placeholders": [
+      {
+        "path": "pyproject.toml",
+        "description": "project.name is 'starter-triton' — rename to match your actual project before publishing to PyPI or internal package registries."
+      }
+    ]
+  },
+  "contextHints": {
+    "commands": [
+      "make install",
+      "make install-dev",
+      "make typecheck",
+      "make lint",
+      "make up",
+      "make health",
+      "make client",
+      "make logs"
+    ],
+    "entrypoints": [
+      "src/client.py",
+      "src/__init__.py",
+      "docker-compose.yml",
+      "model_repository/"
+    ],
+    "preview": {
+      "path": "/v2/health/ready",
+      "port": "8000"
+    },
+    "extensionPoints": [
+      "src/client.py",
+      "model_repository/",
+      "pyproject.toml",
+      "Makefile"
+    ]
+  }
+}

package/registry/layers/framework/unity-web-proxy/files/index.html

@@ -0,0 +1,104 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <title>{{serviceName}}</title>
+    <style>
+      html, body { margin: 0; padding: 0; height: 100%; background: #0b0d10; color: #e7ecf3; font: 14px/1.5 ui-monospace, Menlo, monospace; overflow: hidden; }
+      #unity-container { position: absolute; inset: 0; display: grid; place-items: center; }
+      #unity-canvas { background: #000; display: block; max-width: 100vw; max-height: 100vh; }
+      #unity-loading-bar { position: absolute; left: 50%; bottom: 48px; transform: translateX(-50%); width: 320px; }
+      #unity-progress-bar-empty { width: 100%; height: 6px; background: #1a2130; border-radius: 3px; overflow: hidden; }
+      #unity-progress-bar-full { width: 0%; height: 100%; background: #6ea8ff; transition: width 120ms linear; }
+      #unity-loading-text { margin-top: 8px; text-align: center; font-size: 12px; color: #95a3b8; }
+      #unity-error { position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%); max-width: 520px; padding: 16px 20px; border: 1px solid #ff8080; border-radius: 8px; background: rgba(40, 12, 12, 0.9); color: #ffd8d8; display: none; }
+      #unity-error.visible { display: block; }
+      #unity-error h2 { margin: 0 0 8px; font-size: 16px; color: #ff8080; }
+      #unity-error code { background: #1a0b0b; padding: 2px 6px; border-radius: 4px; font-size: 12px; }
+    </style>
+  </head>
+  <body>
+    <div id="unity-container">
+      <canvas id="unity-canvas" width="1152" height="648" tabindex="-1"></canvas>
+    </div>
+    <div id="unity-loading-bar">
+      <div id="unity-progress-bar-empty"><div id="unity-progress-bar-full"></div></div>
+      <div id="unity-loading-text">Loading Unity build…</div>
+    </div>
+    <div id="unity-error">
+      <h2>Unity build failed to start</h2>
+      <div id="unity-error-message"></div>
+    </div>
+
+    <script>
+      // Loader metadata — update companyName/productName to match Player Settings.
+      const buildUrl = 'Build';
+      const loaderConfig = {
+        dataUrl:            buildUrl + '/Build.data',
+        frameworkUrl:       buildUrl + '/Build.framework.js',
+        codeUrl:            buildUrl + '/Build.wasm',
+        streamingAssetsUrl: 'StreamingAssets',
+        companyName:        'DefaultCompany',
+        productName:        '{{serviceName}}',
+        productVersion:     '0.1.0',
+      };
+
+      const canvas = document.getElementById('unity-canvas');
+      const loadingBar = document.getElementById('unity-loading-bar');
+      const progressFull = document.getElementById('unity-progress-bar-full');
+      const loadingText = document.getElementById('unity-loading-text');
+      const errorBox = document.getElementById('unity-error');
+      const errorMessage = document.getElementById('unity-error-message');
+
+      function showError(msg) {
+        loadingBar.style.display = 'none';
+        errorBox.classList.add('visible');
+        errorMessage.textContent = msg;
+        console.error(msg);
+      }
+
+      function startUnity() {
+        if (typeof createUnityInstance === 'undefined') {
+          showError(
+            'createUnityInstance is undefined — Build/Build.loader.js ' +
+            'did not define the expected Unity API. Verify the editor export.'
+          );
+          return;
+        }
+        createUnityInstance(canvas, loaderConfig, (progress) => {
+          progressFull.style.width = (progress * 100).toFixed(1) + '%';
+          loadingText.textContent = 'Loading Unity build… ' + Math.round(progress * 100) + '%';
+        }).then((unityInstance) => {
+          loadingBar.style.display = 'none';
+          window.__unity = unityInstance;
+        }).catch((err) => {
+          showError('createUnityInstance rejected: ' + (err?.message ?? err));
+        });
+      }
+
+      // SharedArrayBuffer is required for Unity 2021.2+ threaded builds.
+      if (typeof SharedArrayBuffer === 'undefined') {
+        showError(
+          'SharedArrayBuffer is unavailable — the page must be served ' +
+          'with Cross-Origin-Opener-Policy: same-origin and ' +
+          'Cross-Origin-Embedder-Policy: require-corp. ' +
+          'Vite dev is pre-configured; verify your production host.'
+        );
+      } else {
+        // Load Unity's WebGL bootstrap (Build/Build.loader.js) at runtime so
+        // the bundler does not try to resolve it — it is Unity editor output,
+        // not project source.
+        const loaderScript = document.createElement('script');
+        loaderScript.src = buildUrl + '/Build.loader.js';
+        loaderScript.onload = startUnity;
+        loaderScript.onerror = () => showError(
+          'Build/Build.loader.js failed to load. ' +
+          'Drop the Unity WebGL editor export into Build/ first. ' +
+          'Filenames come from Player Settings → Product Name.'
+        );
+        document.body.appendChild(loaderScript);
+      }
+    </script>
+  </body>
+</html>

package/registry/layers/framework/unity-web-proxy/manifest.json

@@ -0,0 +1,63 @@
+{
+  "id": "unity-web-proxy",
+  "description": "Unity web framework layer — index.html with a real Unity loader shim (createUnityInstance, progress bar, early SharedArrayBuffer guard, error surface). Pairs with the unity-web-proxy family files (Vite dev server, tsconfig, batch-mode build script).",
+  "appliesTo": ["unity-web-proxy"],
+  "files": [
+    {
+      "source": "files/index.html",
+      "target": "index.html"
+    }
+  ],
+  "packageDeps": {
+    "devDependencies": {
+      "typescript": "^5.9.0",
+      "vite": "^6.4.2"
+    }
+  },
+  "buildHints": {
+    "whenToUse": "Unity-authored games or interactive content that must ship as a browser-native WebGL build. This layer wires the HTML entry point — the family supplies the Vite dev server, tsconfig, and CI batch-mode build script.",
+    "firstSteps": [
+      "Run `pnpm install` then `pnpm dev` — Vite serves on http://localhost:5173 with the COOP/COEP headers Unity 2021.2+ needs for SharedArrayBuffer.",
+      "Drop the Unity WebGL editor export into `Build/` — you need `Build.loader.js`, `Build.framework.js`, `Build.wasm`, and `Build.data` (filenames reflect Player Settings → Product Name).",
+      "If your Unity product name differs from `Build`, update the `buildUrl` constant and the `<script src>` in `index.html` to match.",
+      "Run `pnpm build` to produce a static `dist/` ready for any CDN — ensure the CDN also sets the COOP/COEP headers or threaded builds will fail."
+    ],
+    "gotchas": [
+      "Unity 2021.2+ WebGL builds require `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` — without them the loader fails with a SharedArrayBuffer error. Vite's dev server is pre-configured; mirror these headers on the production host.",
+      "The loader filename in `index.html` (`Build/Build.loader.js`) is a placeholder matching the default build script output. It must exactly match the filename Unity writes, which is derived from Player Settings → Product Name.",
+      "Brotli/gzip-compressed Unity output (`.br` / `.gz` files) requires the server to send the correct `Content-Encoding` header; most static hosts strip it. Either disable compression in Unity Player Settings or verify headers with `curl -I` after deploy."
+    ],
+    "architectureNotes": [
+      "index.html checks for `SharedArrayBuffer` before calling `createUnityInstance` and surfaces a human-readable error if missing — keeps the COOP/COEP misconfiguration failure mode visible instead of the opaque Unity loader panic.",
+      "The Unity canvas owns full-viewport layout via CSS `inset: 0` on `#unity-container` — do not add CSS transforms to `#unity-canvas` itself, as Unity resizes it via the API and external transforms are ignored."
+    ],
+    "placeholders": [
+      {
+        "path": "index.html",
+        "description": "Loads Unity's WebGL bootstrap from ./Build/Build.loader.js. Update `buildUrl`, the `<script src>`, and the `companyName`/`productName` constants to match your Unity Player Settings after the first editor export."
+      }
+    ]
+  },
+  "contextHints": {
+    "commands": [
+      "pnpm install",
+      "pnpm dev",
+      "pnpm build",
+      "pnpm run build:webgl",
+      "node validate-unity.mjs"
+    ],
+    "entrypoints": [
+      "index.html",
+      "vite.config.ts",
+      "scripts/build-webgl.sh",
+      "Build/"
+    ],
+    "preview": {
+      "path": "/"
+    },
+    "extensionPoints": [
+      "index.html",
+      "scripts/build-webgl.sh"
+    ]
+  }
+}

package/registry/layers/framework/vision-first-agent/files/main.ts

@@ -0,0 +1,143 @@
+// Vision agent browser runtime.
+//
+// Flow: getUserMedia({video}) → <video> live preview → user clicks Capture
+// → offscreen canvas grabs a JPEG frame → POST {image, prompt} to
+// /api/vision → response renders as text + optional bounding boxes on the
+// overlay canvas.
+
+interface Box {
+  label: string
+  x: number
+  y: number
+  w: number
+  h: number
+}
+
+interface VisionResponse {
+  answer: string
+  boxes?: Box[]
+}
+
+const cam = document.getElementById('cam') as HTMLVideoElement
+const overlay = document.getElementById('overlay') as HTMLCanvasElement
+const work = document.getElementById('work') as HTMLCanvasElement
+const askBtn = document.getElementById('ask') as HTMLButtonElement
+const flipBtn = document.getElementById('flip') as HTMLButtonElement
+const q = document.getElementById('q') as HTMLInputElement
+const answer = document.getElementById('answer') as HTMLPreElement
+
+let currentStream: MediaStream | null = null
+let facing: 'user' | 'environment' = 'environment'
+
+async function startCamera(): Promise<void> {
+  // Tear down any previous stream FIRST — the camera LED stays on until
+  // every track is explicitly stopped. Just reassigning srcObject leaks the
+  // device handle.
+  currentStream?.getTracks().forEach((t) => t.stop())
+
+  try {
+    currentStream = await navigator.mediaDevices.getUserMedia({
+      video: {
+        facingMode: { ideal: facing },
+        width: { ideal: 1280 },
+        height: { ideal: 720 },
+      },
+      audio: false,
+    })
+  } catch (err) {
+    const message = err instanceof Error ? err.message : String(err)
+    answer.textContent = `camera denied: ${message}. Serve over HTTPS and allow camera permission.`
+    return
+  }
+
+  cam.srcObject = currentStream
+  // Wait for metadata so videoWidth/videoHeight are populated before
+  // sizing the overlay canvas — 0×0 frames are easy to ship accidentally.
+  await new Promise<void>((resolve) => {
+    if (cam.readyState >= 1) return resolve()
+    cam.addEventListener('loadedmetadata', () => resolve(), { once: true })
+  })
+  overlay.width = cam.videoWidth
+  overlay.height = cam.videoHeight
+}
+
+function captureFrame(): string {
+  const w = cam.videoWidth
+  const h = cam.videoHeight
+  if (!w || !h) throw new Error('camera not ready')
+  work.width = w
+  work.height = h
+  const ctx = work.getContext('2d')
+  if (!ctx) throw new Error('2d context unavailable')
+  ctx.drawImage(cam, 0, 0, w, h)
+  // JPEG quality 0.82 is a sweet spot — indistinguishable from 1.0 to the
+  // human eye, ~4× smaller payload. PNG is never worth it for camera frames.
+  return work.toDataURL('image/jpeg', 0.82)
+}
+
+function drawBoxes(boxes: Box[]): void {
+  const ctx = overlay.getContext('2d')
+  if (!ctx) return
+  ctx.clearRect(0, 0, overlay.width, overlay.height)
+  ctx.lineWidth = Math.max(2, Math.round(overlay.width / 320))
+  ctx.strokeStyle = '#5fff85'
+  ctx.fillStyle = 'rgba(95, 255, 133, 0.85)'
+  ctx.font = `${Math.max(14, Math.round(overlay.width / 48))}px system-ui`
+  for (const b of boxes) {
+    const x = b.x * overlay.width
+    const y = b.y * overlay.height
+    const w = b.w * overlay.width
+    const h = b.h * overlay.height
+    ctx.strokeRect(x, y, w, h)
+    const labelPadding = 4
+    const metrics = ctx.measureText(b.label)
+    const textH = Number.parseInt(ctx.font, 10) + 2
+    ctx.fillStyle = '#5fff85'
+    ctx.fillRect(x, y - textH - labelPadding, metrics.width + labelPadding * 2, textH + labelPadding)
+    ctx.fillStyle = '#000'
+    ctx.fillText(b.label, x + labelPadding, y - labelPadding - 2)
+    ctx.fillStyle = 'rgba(95, 255, 133, 0.85)'
+  }
+}
+
+function clearOverlay(): void {
+  overlay.getContext('2d')?.clearRect(0, 0, overlay.width, overlay.height)
+}
+
+async function ask(): Promise<void> {
+  clearOverlay()
+  askBtn.disabled = true
+  answer.textContent = 'thinking…'
+  try {
+    const image = captureFrame()
+    const prompt = q.value.trim() || 'What am I looking at?'
+    const res = await fetch('/api/vision', {
+      method: 'POST',
+      headers: { 'content-type': 'application/json' },
+      body: JSON.stringify({ image, prompt }),
+    })
+    if (!res.ok) throw new Error(`vision failed: ${res.status}`)
+    const data = (await res.json()) as VisionResponse
+    answer.textContent = data.answer
+    if (data.boxes?.length) drawBoxes(data.boxes)
+  } catch (err) {
+    answer.textContent = `error: ${err instanceof Error ? err.message : String(err)}`
+  } finally {
+    askBtn.disabled = false
+  }
+}
+
+askBtn.addEventListener('click', () => void ask())
+q.addEventListener('keydown', (ev) => {
+  if (ev.key === 'Enter') void ask()
+})
+flipBtn.addEventListener('click', () => {
+  facing = facing === 'environment' ? 'user' : 'environment'
+  void startCamera()
+})
+
+window.addEventListener('pagehide', () => {
+  currentStream?.getTracks().forEach((t) => t.stop())
+})
+
+void startCamera()

package/registry/layers/framework/vision-first-agent/files/vite-env.d.ts

@@ -0,0 +1 @@
+/// <reference types="vite/client" />

package/registry/layers/framework/vision-first-agent/manifest.json

@@ -0,0 +1,47 @@
+{
+  "id": "vision-first-agent",
+  "description": "Camera-agent browser runtime — getUserMedia video preview, canvas frame capture, multimodal LLM POST, overlay rendering of text + optional bounding boxes.",
+  "appliesTo": ["vision-first-agent"],
+  "files": [
+    { "source": "files/main.ts", "target": "src/main.ts" },
+    { "source": "files/vite-env.d.ts", "target": "src/vite-env.d.ts" }
+  ],
+  "packageDeps": {
+    "devDependencies": {
+      "@types/node": "^20.17.0",
+      "typescript": "^5.9.0",
+      "vite": "^6.4.2"
+    }
+  },
+  "buildHints": {
+    "whenToUse": "Any product where the primary input is the device camera: visual QA, barcode/QR readers, OCR tools, 'describe what you see' agents, webcam-based search, live inspection apps.",
+    "firstSteps": [
+      "Run `pnpm install` then `pnpm dev` — Vite serves on the configured port; the browser will prompt for camera permission on first visit.",
+      "The frontend is complete — focus on implementing the `/api/vision` backend described in `api/README.md`. Any multimodal LLM endpoint accepting `{ image: string, prompt: string }` and returning `{ answer: string }` works; `boxes` (bounding boxes) is optional.",
+      "Run `pnpm validate` to verify scaffold wiring without opening a browser — checks DOM IDs, API route, frame-capture call, and TypeScript syntax.",
+      "Run `pnpm build` to emit a static bundle to `dist/` — deploy to any static host; HTTPS is required for `getUserMedia` outside localhost."
+    ],
+    "gotchas": [
+      "With `strict: true` in tsconfig, `document.getElementById()` returns `HTMLElement | null`. Every DOM query in `src/main.ts` uses an explicit `as` cast (e.g. `as HTMLVideoElement`). Without the cast, video/canvas/button-specific properties like `.videoWidth`, `.getContext()`, `.disabled` won't type-check.",
+      "The `/api` proxy in `vite.config.ts` is a dev-only convenience. In production, route `/api/*` through your deployment platform — Cloudflare Workers routes, Vercel rewrites, Nginx `proxy_pass`, etc.",
+      "`<canvas id=\"work\" hidden>` in `index.html` is the off-screen frame-capture surface used by `captureFrame()` in `src/main.ts`. It is invisible by design. Removing it or changing its id breaks the capture flow."
+    ],
+    "architectureNotes": [
+      "All browser logic lives in `src/main.ts` — camera lifecycle, frame capture, API call, and overlay rendering. Fork cleanly from there.",
+      "The hidden `#work` canvas captures frames off-screen; `#overlay` is the visible canvas that draws bounding boxes on top of the `#cam` video element. They serve different roles — don't merge them.",
+      "Bounding box coordinates from the API are normalized 0..1 so they survive display-size rescaling — multiply by `overlay.width`/`overlay.height` to get pixel coordinates."
+    ],
+    "placeholders": [
+      {
+        "path": "src/main.ts",
+        "description": "Bootstrap camera agent. Wire a real `/api/vision` backend (see `api/README.md`). Optionally replace the bounding-box overlay with custom rendering (heat maps, segmentation masks, etc.)."
+      }
+    ]
+  },
+  "contextHints": {
+    "commands": ["pnpm install", "pnpm dev", "pnpm build", "pnpm validate"],
+    "entrypoints": ["src/main.ts", "index.html", "api/README.md"],
+    "preview": { "path": "/" },
+    "extensionPoints": ["src/main.ts", "api/"]
+  }
+}