pi-tldraw 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Zbigniew Siwiec
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,222 @@
+# pi-tldraw
+
+A Pi extension that opens a local tldraw MCP canvas, lets the agent inspect/edit it, and persists canvas snapshots per project folder.
+
+## Install
+
+From npm:
+
+```bash
+pi install npm:pi-tldraw
+```
+
+Pinned:
+
+```bash
+pi install npm:pi-tldraw@0.1.0
+```
+
+Before the npm package is published, you can install from GitHub after a repo/tag exists:
+
+```bash
+pi install git:github.com/<github-owner>/pi-tldraw@v0.1.0
+```
+
+Or quick-run a local checkout without installing:
+
+```bash
+pi -e .
+```
+
+## Runtime requirement: tldraw MCP app
+
+The extension talks to a tldraw MCP server at:
+
+```text
+http://127.0.0.1:8787/mcp
+```
+
+If that endpoint is not reachable and the endpoint is local, the extension tries to auto-start a local tldraw MCP app. Auto-start uses:
+
+1. `TLDRAW_MCP_APP_DIR`, if set.
+2. `./mcp-app` at the package root, if present in the installed package.
+
+For a development checkout of tldraw:
+
+```bash
+export TLDRAW_MCP_APP_DIR=/path/to/tldraw/apps/mcp-app
+```
+
+If you prefer to run the MCP server yourself, start it separately and disable auto-start:
+
+```bash
+export TLDRAW_MCP_AUTO_START=false
+```
+
+The browser-side MCP Apps bridge is different: it is built from source in this repository and shipped as `static/app-bridge-bundle.js`.
+
+## Extension shape
+
+This is a testable Pi extension package:
+
+```text
+pi-tldraw/
+├── src/
+│   ├── index.ts             # Pi extension entry point / registration shell
+│   ├── canvas/              # pure canvas state helpers + workflows
+│   ├── commands/            # pure slash-command parsing
+│   ├── host/                # local browser host adapter
+│   ├── mcp/                 # MCP HTTP client + response parsing
+│   ├── semantic/            # pure canvas semantic rendering layer
+│   ├── server/              # local MCP server lifecycle adapter
+│   ├── store/               # project snapshot persistence
+│   └── ui/                  # Pi status widget adapter
+├── test/                    # pure helper/workflow/unit tests
+├── bridge/                   # source for the browser-side MCP Apps bridge bundle
+├── scripts/                  # reproducible build / verification / local e2e helpers
+├── static/app-bridge-bundle.js
+├── static/app-bridge-bundle.meta.json
+├── static/host.html
+├── mcp-app/                 # packaged tldraw MCP app runtime when release assembly provides it
+├── package.json             # Pi package metadata
+└── tsconfig.json
+```
+
+Pi discovers the extension via:
+
+```json
+{
+  "pi": {
+    "extensions": ["./src/index.ts"]
+  }
+}
+```
+
+## Build and release contract
+
+- Users installing from npm should receive prebuilt browser assets; they should not compile the bridge during `pi install`.
+- Maintainers build the bridge in CI from the source under `bridge/` before publishing.
+- The npm tarball includes both source (`bridge/`, `scripts/`) and the generated artifact (`static/app-bridge-bundle.js`, `static/app-bridge-bundle.meta.json`).
+- Git installs work from tagged commits because the generated static artifact is committed and verified by CI.
+- The runtime contract remains the same: the extension auto-starts a local `mcp-app/` when the release package contains it, or uses `TLDRAW_MCP_APP_DIR` / a manually started server.
+
+## Normal UX
+
+Start or restore the current project canvas:
+
+```text
+/tldraw open
+```
+
+Then work in the browser canvas. Edits are autosaved into the current project folder:
+
+```text
+.pi/tldraw-canvases/index.json
+.pi/tldraw-canvases/<canvasId>.json
+```
+
+You should not need a manual save during normal use.
+
+## Useful commands
+
+```text
+/tldraw open [canvasId]      # Open/restore canvas; creates one if needed
+/tldraw host                 # Show host/server/autosave diagnostics
+/tldraw canvases             # List saved project canvases
+/tldraw current              # Show current project canvas id
+/tldraw restart              # Restart local MCP app server
+/tldraw save                 # Manual checkpoint; refuses unsafe blank overwrite
+/tldraw save!                # Force manual empty overwrite
+/tldraw reset                # Reset MCP HTTP session
+```
+
+## Agent tools
+
+- `tldraw_canvas_open` — open/restore the local browser canvas host.
+- `tldraw_diagram_tips` — get minimalist drawing guidance before creating or revising a diagram.
+- `tldraw_canvas_export` — export the current selection or whole canvas as a PNG/SVG image for visual feedback.
+- `tldraw_canvas_scene` — compact semantic canvas view; falls back to saved snapshots.
+- `tldraw_canvas_state` — inspect live canvas state and optionally save it.
+- `tldraw_canvas_exec` — execute JavaScript against the live tldraw editor.
+- `tldraw_search` — inspect the tldraw Editor API via the MCP search tool.
+- `tldraw_status` — check MCP server tools/resources.
+
+## Persistence model
+
+The durable source of truth is project-local JSON:
+
+```text
+<project>/.pi/tldraw-canvases/<canvasId>.json
+```
+
+The MCP server checkpoint store is treated as transient runtime state. If the browser port, MCP session, or Wrangler state changes, `/tldraw open` restores from the project snapshot.
+
+Safety rules:
+
+- Autosave and manual save refuse to overwrite a non-empty saved snapshot with an empty live canvas.
+- Forced empty overwrite is available with `/tldraw save!`.
+- Previous snapshots are copied to `.pi/tldraw-canvases/history/<canvasId>/` before overwrite.
+
+## Development
+
+```bash
+npm install
+npm run build       # builds static/app-bridge-bundle.js from bridge/app-bridge-entry.js
+npm run check
+npm run pack:dry
+```
+
+The bridge bundle is intentionally prebuilt for users, but maintainers build it from source with `npm run build:bridge`. CI rebuilds it and fails if the committed artifact drifts from source.
+
+The packaged MCP app is assembled from a built tldraw checkout and records OSS provenance in `mcp-app/PI_TLDRAW_PROVENANCE.json`:
+
+```bash
+# source checkout must already have apps/mcp-app/dist/* built
+export TLDRAW_MCP_APP_SOURCE_DIR=/path/to/tldraw/apps/mcp-app
+npm run assemble:mcp-app
+npm run verify:mcp-app
+npm run verify:mcp-app-source
+```
+
+The source contract is declared in `mcp-app-source.json`: upstream tldraw repo, pinned commit, app path, build commands, and patch files under `patches/tldraw-mcp-app/`. The current packaged app is traceable to that pinned source plus `patches/tldraw-mcp-app/001-pi-runtime.patch`.
+
+The packaged `mcp-app/` keeps the same runtime contract as before (`cd mcp-app && yarn dev`) but uses prebuilt `dist/` assets so users do not rebuild tldraw during install/startup.
+
+For maintainers who want to rebuild from the pinned upstream source directly:
+
+```bash
+npm run assemble:mcp-app:pinned
+```
+
+This clones the pinned tldraw commit, applies declared patches, runs the manifest build commands, and assembles `mcp-app/`.
+
+Release checklist:
+
+```bash
+npm ci
+export TLDRAW_MCP_APP_SOURCE_DIR=/path/to/tldraw/apps/mcp-app
+npm run assemble:mcp-app
+npm run verify:mcp-app-source
+npm run build
+npm run check
+npm run e2e:packaged-mcp-app
+npm run pack:dry
+```
+
+GitHub Actions runs the same build/check path before publishing with npm provenance.
+
+To smoke-test against a running tldraw MCP server:
+
+```bash
+# optional: defaults to http://127.0.0.1:8787/mcp
+export TLDRAW_MCP_URL=http://127.0.0.1:8787/mcp
+npm run e2e:mcp
+```
+
+To smoke-test the packaged MCP app itself on a temporary port:
+
+```bash
+npm run e2e:packaged-mcp-app
+```
+
+Keep logic that does not need Pi APIs in separate modules, such as `src/store/project-store.ts`, so it can be unit-tested without mocking Pi.

package/bridge/app-bridge-entry.js

@@ -0,0 +1,6 @@
+import { AppBridge, PostMessageTransport } from '@modelcontextprotocol/ext-apps/app-bridge'
+
+// Browser host entry point for Pi's local tldraw iframe host.
+// Keep this intentionally small: the build step owns bundling MCP Apps bridge
+// dependencies into static/app-bridge-bundle.js for zero-build installs.
+globalThis.McpAppsBridge = { AppBridge, PostMessageTransport }

package/mcp-app/LICENSE.md

@@ -0,0 +1,9 @@
+MIT License
+
+Copyright (c) 2024 tldraw Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/mcp-app/PI_TLDRAW_PROVENANCE.json

@@ -0,0 +1,32 @@
+{
+  "artifact": "mcp-app/",
+  "builder": "scripts/assemble-mcp-app.mjs",
+  "source": {
+    "mode": "local",
+    "repo": "https://github.com/tldraw/tldraw.git",
+    "commit": "4b0cfc539e074217e1e248461afa596fc7d02040",
+    "appPath": "apps/mcp-app",
+    "patches": [
+      {
+        "path": "patches/tldraw-mcp-app/001-pi-runtime.patch",
+        "sha256": "6bb0b7b7e9a2b9379bfe13800e4407c346b307e3e241d3c4ffe2f1876d1af894"
+      }
+    ],
+    "build": [
+      {
+        "cwd": ".",
+        "command": "yarn install --immutable"
+      },
+      {
+        "cwd": "apps/mcp-app",
+        "command": "yarn build"
+      }
+    ],
+    "sourceGitCommit": "4b0cfc539e074217e1e248461afa596fc7d02040"
+  },
+  "dist": {
+    "mcp-app.html": "24af4cfaef84460f172dec501803960bf161f2faeab9c5eff8c6b0b3ace83c67",
+    "editor-api.json": "7298c0069c700f4b5db94a31ab3fbf742a64dc97442cf93f63616ab029153620",
+    "method-map.json": "e060b2860d245d07bddd9f2ad41398dd733328eba48234058854ac03f1bb828e"
+  }
+}

package/mcp-app/README.md

@@ -0,0 +1,129 @@
+# tldraw MCP app
+
+This is the tldraw MCP app. It exposes an interactive tldraw canvas to AI agents via the [Model Context Protocol app specification](https://github.com/modelcontextprotocol/ext-apps/), so you can work in tldraw with agents in any MCP client that supports the MCP app spec.
+
+## Architecture
+
+The app has two parts: a **server** and a **widget**.
+
+### Server
+
+The server runs in Cloudflare Workers via `src/worker.ts`, using a Durable Object (`TldrawMCP`) backed by SQLite for persistent checkpoint storage.
+
+It exposes:
+
+- `search` — query the extracted Editor API spec in a sandboxed dynamic worker
+- `exec` — execute JavaScript against the live editor in the widget via a pending-request callback bridge
+- `_exec_callback` — app-only tool the widget calls to resolve a pending `exec` request
+- `save_checkpoint` / `read_checkpoint` — app-only tools used by the widget for checkpoint persistence
+
+### Widget
+
+The widget is a React app (`src/widget/mcp-app.tsx`) that renders a full tldraw canvas inside the MCP host's iframe.
+
+When the AI calls `exec`, the server creates a pending request and the widget picks it up, runs the code through a focused editor proxy (`src/widget/focused/`) that translates between an AI-friendly shape format (simple string IDs, flat `_type` shapes) and tldraw's internal `TLShape`/`TLShapeId` types, then calls `_exec_callback` to resolve the pending request with the result. Canvas state is checkpointed to the Durable Object's SQLite database and to the browser's local storage.
+
+## Developing
+
+### Prerequisites
+
+The widget build depends on generated files (`editor-api.json`, `method-map.json`) that are extracted from the editor's TypeScript declarations. Before you can develop or build the mcp-app, you need to build the core packages first:
+
+```bash
+# from the repo root
+yarn build
+```
+
+This produces the `.tsbuild/` output that `yarn extract-api` reads from. The `build` and `dev` scripts run `extract-api` automatically, so you don't need to call it separately.
+
+### Package scripts
+
+Run all commands from `apps/mcp-app`.
+
+| Command           | What it does                                                                                       |
+| ----------------- | -------------------------------------------------------------------------------------------------- |
+| `yarn build`      | Build the widget HTML                                                                              |
+| `yarn dev`        | Build widget + start local Cloudflare worker (HTTP MCP on `localhost:8787`)                        |
+| `yarn dev:tunnel` | Build widget + start a Cloudflare tunnel + local worker with `WORKER_ORIGIN` set to the tunnel URL |
+| `yarn deploy`     | Build widget + deploy the Cloudflare worker to production                                          |
+
+`yarn dev:tunnel` requires the `cloudflared` CLI to be installed on your machine.
+
+The worker defaults to production-safe behavior in `wrangler.toml`, including setting `MCP_IS_DEV="false"`. Local HTTP dev scripts override that with `MCP_IS_DEV=true` so local Claude/ChatGPT connectors suppress `ui.domain` while production deployments keep it enabled.
+
+### Cursor setup
+
+Add these two servers in `~/.cursor/mcp.json`:
+
+```json
+{
+	"mcpServers": {
+		"tldraw": {
+			"transport": "http",
+			"url": "https://tldraw-mcp-app.tldraw.workers.dev/mcp"
+		},
+		"tldraw-local": {
+			"command": "npx",
+			"args": ["-y", "mcp-remote", "http://127.0.0.1:8787/mcp"]
+		}
+	}
+}
+```
+
+### Claude Desktop local setup
+
+For local Claude Desktop development, use `claude_desktop_config.json` with the local HTTP server:
+
+```json
+{
+	"mcpServers": {
+		"tldraw-local": {
+			"command": "npx",
+			"args": ["-y", "mcp-remote", "http://127.0.0.1:8787/mcp"]
+		}
+	}
+}
+```
+
+### Claude Desktop remote setup
+
+If you'd like to try the remote MCP server in Claude Desktop, use the in-app connector flow rather than adding the production URL to `claude_desktop_config.json`.
+
+1. Open Claude Desktop
+2. In the sidebar, go to **Customize**
+3. Open **Connectors**
+4. Click the button to add a connector, then choose **Add custom connector**
+5. Give it a name such as `tldraw`
+6. Paste `https://tldraw-mcp-app.tldraw.workers.dev/mcp` as the server URL
+
+The **Add custom connector** option is not available on the free plan, so you may need Max or another paid plan.
+
+If you need Notion access in Claude Desktop, use the Notion MCP connector for that separately.
+
+### ChatGPT local dev
+
+ChatGPT requires an HTTPS origin, so you need a Cloudflare tunnel. You must be an admin of your OpenAI org/workspace to do local dev.
+
+1. Run `yarn dev:tunnel` in `apps/mcp-app`
+2. It prints a `https://...trycloudflare.com` tunnel URL
+3. In ChatGPT web (not the desktop app), go to **Apps** and add your app using that tunnel URL
+4. You can then test in both ChatGPT web and the desktop or mobile apps
+
+`dev:tunnel` automatically wires `WORKER_ORIGIN` to the tunnel URL and sets `MCP_IS_DEV=true` for the local worker.
+
+### Iteration loop
+
+1. Make code changes in `apps/mcp-app`
+2. Run the relevant script (`dev` or `dev:tunnel`)
+3. Disconnect and reconnect the MCP server in your client (or reload the page/app)
+4. When making widget changes, make sure to rebuild, either by running `yarn build` or rerunning any of the dev scripts.
+
+Reconnecting the server after changes is the most reliable way to pick up new code, especially when the widget HTML changes.
+
+## Contact
+
+Find us on Twitter/X at [@tldraw](https://twitter.com/tldraw).
+
+## Community
+
+Have questions, comments or feedback? [Join our discord](https://discord.tldraw.com/?utm_source=github&utm_medium=readme&utm_campaign=sociallink). For the latest news and release notes, visit [tldraw.dev](https://tldraw.dev).

package/mcp-app/dev-tunnel.sh

@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+# Starts a cloudflared tunnel + wrangler dev with the tunnel URL as WORKER_ORIGIN.
+# Usage: ./dev-tunnel.sh
+set -euo pipefail
+
+PORT="${PORT:-8787}"
+LOGFILE=$(mktemp)
+
+# Build widget first
+echo "Building widget..."
+yarn build:widget
+
+# Start cloudflared in background, capture the tunnel URL from stderr
+cloudflared tunnel --url "http://localhost:$PORT" 2>"$LOGFILE" &
+CF_PID=$!
+
+cleanup() {
+  echo "Stopping cloudflared (pid $CF_PID)..."
+  kill "$CF_PID" 2>/dev/null || true
+  rm -f "$LOGFILE"
+}
+trap cleanup EXIT
+
+# Wait for cloudflared to print the tunnel URL
+echo "Waiting for tunnel URL..."
+TUNNEL_URL=""
+for i in $(seq 1 30); do
+  TUNNEL_URL=$(grep -o 'https://[a-z0-9-]*\.trycloudflare\.com' "$LOGFILE" | head -1 || true)
+  if [ -n "$TUNNEL_URL" ]; then
+    break
+  fi
+  sleep 1
+done
+
+if [ -z "$TUNNEL_URL" ]; then
+  echo "ERROR: Failed to get tunnel URL after 30s. cloudflared log:"
+  cat "$LOGFILE"
+  exit 1
+fi
+
+echo ""
+echo "==================================="
+echo "Tunnel URL: $TUNNEL_URL"
+echo "==================================="
+echo ""
+echo "Update claude_desktop_config.json to use:"
+echo "  \"args\": [\"-y\", \"mcp-remote\", \"$TUNNEL_URL/mcp\"]"
+echo ""
+
+# Start wrangler with the tunnel URL as WORKER_ORIGIN in dev mode
+exec wrangler dev --port "$PORT" --var "WORKER_ORIGIN:$TUNNEL_URL" --var "MCP_IS_DEV:true"