@letta-ai/letta-code 0.31.6 → 0.31.8

package/skills/browser-use/SKILL.md

@@ -0,0 +1,363 @@
+---
+name: browser-use
+description: Control a real browser to navigate pages, click, type, fill forms, inspect rendered UI, take screenshots, or record video. Load only when the user asks to open or automate a browser, interact with or test rendered page UI, scrape a site that needs browser execution, or capture a browser screenshot or video. Do not load for backend logs, traces, API or stream events, source-code inspection, or plain HTTP or web research that does not require a browser.
+---
+
+# Browser Use with CDP
+
+Drive the browser through its native Chrome DevTools Protocol over the
+remote-debugging WebSocket. This works with zero dependencies: launch the
+browser with `--remote-debugging-port`, then talk JSON over `fetch` and the
+built-in `WebSocket` global (available in Bun and Node ≥ 22 — no `ws` package).
+
+If the project already has Playwright or Puppeteer installed, using it is
+usually simpler — reach for raw CDP when no automation library is available,
+when protocol-level control is needed, or when recording a deterministic
+visual demo.
+
+Protocol reference: https://chromedevtools.github.io/devtools-protocol/.
+The running browser's exact schema is at `http://127.0.0.1:<port>/json/protocol`;
+tip-of-tree docs can differ from the installed version.
+
+## Managed cloud sandbox default: visible browser
+
+When running in a cloud sandbox, default every browser task to the visible
+managed desktop, even when the user did not explicitly ask to watch. Most
+browser tasks exist because plain HTTP is not enough; a headless browser is
+more likely to trigger bot protection and gives the user no way to observe or
+take over. This matters especially for clicking or typing, forms, sign-in,
+checkout/payment, CAPTCHAs or bot protection, and user handoff.
+
+For the first managed-sandbox browser window, use the skill's launcher instead
+of assembling Chrome, DISPLAY, or Xvfb commands yourself:
+
+```bash
+/root/.letta/cloud-skills/browser-use/scripts/open-visible-browser.sh 'https://example.com'
+```
+
+It starts the managed desktop and launches Chrome through the persistent Cua
+Driver with its required root flag. Then load `computer-use` for visible
+interaction. If protocol-level control is necessary, use Cua Driver's explicit
+`browser_prepare` flow after binding the exact visible window; do not pass
+remote-debugging flags through `launch_app`. The launcher exits zero only
+after Cua Driver reports an on-screen browser window. If it exits nonzero, stop
+and report the launch failure instead of claiming the browser opened.
+
+When the user asks to review, watch, or take over, leave that browser window
+open after the task. Do not kill or close it before replying.
+
+1. Run `start-letta-desktop` and use its exit status as the result. Warnings
+   from optional services do not mean startup failed when the command exits 0.
+   If it exits nonzero, stop and report that the managed desktop is
+   unavailable. Never create another Xvfb, VNC server, or private display: the
+   Computer viewer only shows the managed desktop.
+2. Load `computer-use`, inspect the managed desktop, and use Cua Driver to
+   operate an existing Chrome window or launch Chrome there. When launching,
+   round-trip Chrome's `launch_path` from `cua-driver call list_apps '{}'
+   instead of rebuilding it; the managed launch path carries required flags.
+   For forms, sign-in, checkout, CAPTCHA, and bot-protected pages, keep using
+   Cua Driver so the interaction remains visible and available for user
+   takeover.
+3. Use CDP only when protocol-level inspection or deterministic automation is
+   needed. Bind the exact visible browser window with Cua Driver, then use its
+   explicit `browser_prepare` flow. Do not pass remote-debugging flags
+   through `launch_app`. Include `--no-sandbox` when running Chrome as
+   root. Do not add `--headless` or override `DISPLAY`.
+4. Use headless mode only for work the user explicitly wants in the background
+   and that cannot require interaction or handoff, such as read-only scraping,
+   CI, or screenshot/PDF generation.
+5. Verify the result through the managed desktop window (Cua Driver window
+   state or screenshot), not only through DOM output or a screenshot from a
+   separate process.
+
+A headless page does not satisfy a request to open or reopen a site in the
+user-visible browser.
+
+## Workflow
+
+1. Find a Chromium-based browser (below). If none exists, see "No Chrome installed".
+2. In a managed cloud sandbox, follow the visible-browser default above.
+   Otherwise, launch with a dedicated profile and remote debugging. Never
+   attach to the user's normal profile unless explicitly asked.
+3. Discover targets via `/json/list`; pick the `"page"` target by URL or title.
+4. Connect to its `webSocketDebuggerUrl` and enable only the domains you need
+   (usually `Page`, `Runtime`, `DOM`, `Input`; add `Network`, `Log` when debugging).
+5. Inspect before acting: find elements by accessible name, label, text, role,
+   stable ID, or placeholder — not generated classes or child indexes.
+6. Act through `Input.*` for user-like interactions; use `Runtime.evaluate` for
+   inspection, coordinate math, and setup with no meaningful user interaction.
+7. Wait on observable state, never fixed sleeps alone.
+8. Verify the result (DOM state, URL, screenshot, console/network events).
+9. Clean up temporary background work: stop screencasts and close the
+   WebSocket. Kill a browser you launched only when the user did not ask to
+   review, watch, or take over the visible window.
+
+## Finding the browser
+
+Any Chromium-based browser supports CDP (Chrome, Chromium, Edge, Brave). Probe
+in order:
+
+```bash
+# macOS
+for c in "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
+         "/Applications/Chromium.app/Contents/MacOS/Chromium" \
+         "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge" \
+         "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"; do
+  [ -x "$c" ] && { echo "$c"; break; }
+done
+
+# Linux
+for c in google-chrome google-chrome-stable chromium chromium-browser microsoft-edge brave-browser; do
+  command -v "$c" && break
+done
+```
+
+On Windows, check `%ProgramFiles%\Google\Chrome\Application\chrome.exe`,
+`%ProgramFiles(x86)%\...`, `%LocalAppData%\Google\Chrome\Application\chrome.exe`,
+and the same patterns for `Microsoft\Edge`.
+
+### No Chrome installed
+
+Any browser found by the probe above works identically — use it. If truly no
+Chromium-based browser exists, do not install or download one automatically.
+Tell the user that browser use requires Chrome or another Chromium-based
+browser and recommend either:
+
+1. Install Chrome on the current computer, then retry the browser task.
+2. Teleport the conversation back to its Cloud sandbox, where the managed
+   browser is already installed.
+
+Wait for the user to choose. Do not silently replace the browser task with
+plain HTTP or claim browser automation succeeded.
+
+## Launching
+
+Outside a managed cloud sandbox, use a disposable profile and a fixed port.
+Chrome refuses to run as root without `--no-sandbox`, so add that flag when
+`id -u` is 0:
+
+```bash
+chrome_args=( \
+  --remote-debugging-port=9222 \
+  --user-data-dir=/tmp/cdp-profile \
+  --window-size=1440,900 \
+  --force-device-scale-factor=1 \
+  --no-first-run \
+  --no-default-browser-check \
+)
+[ "$(id -u)" -eq 0 ] && chrome_args+=(--no-sandbox)
+"$CHROME" "${chrome_args[@]}" https://example.com
+```
+
+Outside a managed cloud sandbox, add `--headless=new` only for explicitly
+invisible work or when no display exists. In a managed cloud sandbox, follow
+the visible-browser rule above and never replace its managed display with a
+private one.
+With `--remote-debugging-port=0`, read the chosen port from
+`<user-data-dir>/DevToolsActivePort`. Launch in the background and poll
+`http://127.0.0.1:9222/json/version` until it responds.
+
+HTTP endpoints: `/json/version` (browser metadata + browser-level WebSocket URL),
+`/json/list` (targets), `PUT /json/new?<url>` (open tab),
+`/json/activate/<id>`, `/json/close/<id>`, `/json/protocol` (schema).
+
+Attach to the **page** target for `Page`/`DOM`/`Runtime`/`Input` work; use the
+**browser** target only for browser-wide commands (target control, downloads,
+browser contexts).
+
+## Minimal CDP client
+
+CDP messages are JSON with monotonically increasing request ids. Run with `bun`:
+
+```ts
+const targets = await (await fetch("http://127.0.0.1:9222/json/list")).json();
+const target = targets.find((t: any) => t.type === "page");
+if (!target) throw new Error("No page target");
+
+const ws = new WebSocket(target.webSocketDebuggerUrl);
+await new Promise((resolve, reject) => {
+  ws.onopen = resolve;
+  ws.onerror = reject;
+});
+
+let nextId = 0;
+const pending = new Map();
+ws.onmessage = (event) => {
+  const msg = JSON.parse(String(event.data));
+  if (!msg.id) return handleEvent(msg); // Page.loadEventFired, Log.entryAdded, ...
+  const p = pending.get(msg.id);
+  if (!p) return;
+  pending.delete(msg.id);
+  msg.error ? p.reject(new Error(JSON.stringify(msg.error))) : p.resolve(msg.result);
+};
+ws.onclose = () => {
+  for (const p of pending.values()) p.reject(new Error("socket closed"));
+  pending.clear();
+};
+
+function send(method: string, params: object = {}): Promise<any> {
+  return new Promise((resolve, reject) => {
+    const id = ++nextId;
+    pending.set(id, { resolve, reject });
+    ws.send(JSON.stringify({ id, method, params }));
+  });
+}
+
+await send("Page.enable");
+await send("Runtime.enable");
+```
+
+Navigations destroy execution contexts and can invalidate in-flight
+`Runtime.evaluate` calls; retry after observing the new document.
+
+## Inspecting the page
+
+Start with a concise UI inventory:
+
+```ts
+const result = await send("Runtime.evaluate", {
+  expression: `JSON.stringify({
+    buttons: [...document.querySelectorAll('button')].map((el) => ({
+      text: el.innerText.trim(), aria: el.getAttribute('aria-label'), title: el.title
+    })).filter((x) => x.text || x.aria || x.title),
+    inputs: [...document.querySelectorAll('input, textarea, [contenteditable=true]')].map((el) => ({
+      tag: el.tagName, type: el.type, placeholder: el.placeholder,
+      aria: el.getAttribute('aria-label'), value: el.value
+    }))
+  })`,
+  returnByValue: true,
+});
+```
+
+Use `awaitPromise: true` for async expressions and `userGesture: true` when the
+page requires user activation. Treat `exceptionDetails` in the result as an
+error even though the CDP command itself succeeded.
+
+`document.querySelector` does not cross shadow boundaries — traverse open
+shadow roots explicitly; for closed shadow roots or remote-object work use the
+`DOM` domain (`DOM.getDocument`, `DOM.querySelector`, `DOM.getBoxModel`).
+
+## Clicking and typing
+
+Compute coordinates in CSS pixels immediately before acting, then send native
+input events:
+
+```ts
+async function point(expr: string) {
+  const r = await send("Runtime.evaluate", {
+    expression: `(() => {
+      const el = ${expr};
+      if (!el) return null;
+      el.scrollIntoView({ block: 'center', inline: 'center' });
+      const b = el.getBoundingClientRect();
+      return { x: b.left + b.width / 2, y: b.top + b.height / 2 };
+    })()`,
+    returnByValue: true,
+  });
+  if (!r.result.value) throw new Error(`Element not found: ${expr}`);
+  return r.result.value;
+}
+
+async function click(expr: string) {
+  const { x, y } = await point(expr);
+  await send("Input.dispatchMouseEvent", { type: "mouseMoved", x, y });
+  await send("Input.dispatchMouseEvent", { type: "mousePressed", x, y, button: "left", buttons: 1, clickCount: 1 });
+  await send("Input.dispatchMouseEvent", { type: "mouseReleased", x, y, button: "left", buttons: 0, clickCount: 1 });
+}
+```
+
+Focus an editable element (click it), then insert text:
+
+```ts
+await click(`document.querySelector('input[aria-label="Search"]')`);
+await send("Input.insertText", { text: "search terms" });
+```
+
+Use `Input.insertText` for text and Unicode; use paired `Input.dispatchKeyEvent`
+(`keyDown` + `keyUp` with `key`, `code`, `windowsVirtualKeyCode`) for Enter,
+Escape, arrows, Tab, and shortcuts. Modifier bits: Alt=1, Ctrl=2, Meta=4, Shift=8.
+
+Native `<select>` and framework-controlled inputs may need the prototype setter
+plus bubbling events:
+
+```ts
+const setter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value").set;
+setter.call(input, "new value");
+input.dispatchEvent(new Event("input", { bubbles: true }));
+input.dispatchEvent(new Event("change", { bubbles: true }));
+```
+
+Prefer real `Input.*` events for the behavior being demonstrated or tested;
+direct DOM mutation is fine for deterministic setup and inspection.
+
+## Waiting reliably
+
+A returned command does not mean the action completed. Poll the state that
+proves completion:
+
+```ts
+async function waitFor(expr: string, timeoutMs = 30_000) {
+  const start = Date.now();
+  while (Date.now() - start < timeoutMs) {
+    const r = await send("Runtime.evaluate", { expression: `Boolean(${expr})`, returnByValue: true });
+    if (r.result.value) return;
+    await new Promise((res) => setTimeout(res, 250));
+  }
+  throw new Error(`Timed out waiting for ${expr}`);
+}
+
+await waitFor(`document.body.innerText.includes('Saved')`);
+```
+
+For navigation, wait on `Page.loadEventFired` or a lifecycle `networkIdle`
+event — but SPA route changes may emit neither, so prefer the UI condition
+that actually matters.
+
+## Screenshots, PDF, and video
+
+```ts
+const shot = await send("Page.captureScreenshot", { format: "png" });
+await Bun.write("screenshot.png", Buffer.from(shot.data, "base64"));
+```
+
+`captureBeyondViewport: true` for full-page; `Page.getLayoutMetrics` + `clip`
+for exact regions; `Page.printToPDF` for PDFs. For video recording with
+`Page.startScreencast` and demo-polish tips, read
+[references/recording.md](references/recording.md).
+
+## Debugging failures
+
+Enable `Network` and `Log`, then watch `Network.requestWillBeSent`,
+`Network.responseReceived`, `Network.loadingFailed`, `Runtime.consoleAPICalled`,
+`Runtime.exceptionThrown`, and `Log.entryAdded`. Fetch bodies with
+`Network.getResponseBody`. Never log authorization headers, cookies, API keys,
+passwords, or response bodies containing secrets — scrub before returning
+output to context.
+
+Common failure modes:
+
+- **ECONNREFUSED on the port**: browser exited, wrong port, or debugging not
+  enabled — check `/json/version` first.
+- **No matching target**: inspect `/json/list`; match by URL/title, don't take
+  the first page blindly.
+- **Execution context destroyed**: the page navigated; wait for the new
+  document and re-evaluate.
+- **Click misses an existing element**: scroll into view and recalculate the
+  box immediately before dispatching.
+- **Typed text doesn't stick in a controlled input**: focus + `Input.insertText`,
+  or the prototype-setter pattern above.
+- **Opening DevTools disconnects automation**: embedded DevTools can detach
+  other clients — don't open DevTools during a run.
+- **Page commands fail on the browser endpoint**: attach to the page target.
+
+## Safety
+
+Browser automation acts with the user's browser authority.
+
+- Use a disposable profile by default.
+- Do not submit purchases, publish content, send messages, delete data, or
+  accept consequential dialogs without explicit authorization.
+- Do not extract saved passwords, tokens, cookies, or unrelated browsing data.
+- Stay within the requested origin and workflow.
+- Keep the remote-debugging listener on loopback (`127.0.0.1`) unless the user
+  explicitly needs remote access and has authentication in place.

package/skills/browser-use/references/recording.md

@@ -0,0 +1,60 @@
+# Recording video with Page.startScreencast
+
+Screencast frames arrive when Chrome paints, not at a constant frame rate.
+Acknowledge every frame immediately or Chrome stops sending them:
+
+```ts
+let latestFrame: Buffer | undefined;
+
+function handleEvent(msg: any) {
+  if (msg.method !== "Page.screencastFrame") return;
+  latestFrame = Buffer.from(msg.params.data, "base64");
+  void send("Page.screencastFrameAck", { sessionId: msg.params.sessionId });
+}
+
+await send("Page.startScreencast", {
+  format: "jpeg",
+  quality: 88,
+  maxWidth: 1440,
+  maxHeight: 900,
+  everyNthFrame: 1,
+});
+```
+
+To produce a standard constant-frame-rate video, sample `latestFrame` on a
+fixed timer into numbered JPEGs while the flow runs, then encode:
+
+```ts
+let frameNo = 0;
+const sampler = setInterval(() => {
+  if (latestFrame) {
+    void Bun.write(`frames/frame-${String(frameNo++).padStart(6, "0")}.jpg`, latestFrame);
+  }
+}, 100); // 10 fps
+// ... perform the flow ...
+clearInterval(sampler);
+await send("Page.stopScreencast");
+```
+
+```bash
+ffmpeg -y -framerate 10 -i frames/frame-%06d.jpg \
+  -c:v libx264 -crf 18 -pix_fmt yuv420p -movflags +faststart demo.mp4
+```
+
+- Seed `latestFrame` with a `Page.captureScreenshot` before starting so an
+  idle opening scene is not blank.
+- Stop the screencast in a `finally` block.
+- Verify with `ffprobe`, extract a few representative frames with `ffmpeg`,
+  and inspect them before reporting success.
+
+## Demo polish
+
+- Fix the viewport size and `--force-device-scale-factor=1`.
+- Use a clean profile and reset app state before the final take.
+- Rehearse the entire flow once before recording.
+- Add a visible synthetic cursor (an absolutely-positioned element moved via
+  `Runtime.evaluate`) — CDP frames do not include the OS pointer.
+- Pause briefly after meaningful actions so viewers can follow.
+- Never expose credentials: use a dedicated non-sensitive demo token and mask
+  password fields.
+- End on the completed result for several seconds.

package/skills/letta-guide/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: letta-guide
-description: Answer questions about Letta itself from the official documentation. Use whenever the user asks how Letta works, what Letta (or you) can do, or how to set up or configure providers, models, channels, skills, memory, schedules, permissions, self-hosting, pricing, or billing — any "how do I…" or "can Letta…" question about the Letta product. Fetch the docs before answering; never answer Letta product questions from memory alone.
+description: Read the official Letta documentation (docs.letta.com) through its cached, ETag-checked fetch route. Load before ANY docs.letta.com retrieval — answering how Letta works, what Letta (or you) can do, setting up providers, models, channels, skills, memory, schedules, permissions, self-hosting, pricing, or billing, AND looking up Letta API, Agent SDK, or Letta Code reference while writing code. Do not use fetch_webpage or web_search on docs.letta.com; this skill's helper is the docs route. Never answer Letta product questions from memory alone.
 ---
 
 # Letta Guide
@@ -8,7 +8,11 @@ description: Answer questions about Letta itself from the official documentation
 You are running inside Letta, but your training data about Letta's commands,
 flags, settings, UI, pricing, and providers is out of date. Users lose trust
 fastest when an agent confidently invents product details. This skill defines
-how to answer questions about Letta correctly.
+how to read the Letta docs correctly — both when answering questions about
+Letta and when looking up API, Agent SDK, or Letta Code reference during
+development. Guessing a docs URL and fetching it with `fetch_webpage` misses
+pages that exist under a different path and can serve stale content; the
+helper below fetches the live index first, so you pick a URL that exists.
 
 ## Source route (in order)
 

package/skills/self-configuration/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: self-configuration
-description: Inspect or modify Letta Code's own memory, model, context window, system prompt, compaction, permissions, toolsets, mods, skills, channels, schedules, agent secrets, and local runtime settings. Use when the user asks how this agent or conversation is configured, or asks you to change how you behave or how the harness runs you.
+description: Inspect or modify Letta Code's own memory, model, context window, system prompt, compaction, permissions, toolsets, mods, skills, channels, schedules, agent secrets, and local runtime settings. Use when the user asks how this agent or conversation is configured, asks you to change how you behave or how the harness runs you, or renames you.
 license: MIT
 ---
 
@@ -158,6 +158,8 @@ npx tsx <SKILL_DIR>/scripts/update-agent-settings.ts \
 
 Name and description are agent-level metadata. Do not pass them with `--target conversation`. Values must be non-empty; the helper does not clear metadata by accident.
 
+When the user renames you, this patch is the authoritative change — editing a name written in persona memory does not change the agent's actual name. Do both: patch the agent name here, then update any memory file that states your name so they agree.
+
 ```bash
 npx tsx <SKILL_DIR>/scripts/update-agent-settings.ts \
   --target agent \

package/dist/types/utils/system-prompt-size.d.ts.map

@@ -1 +0,0 @@
-{"version":3,"file":"system-prompt-size.d.ts","sourceRoot":"","sources":["../../../src/utils/system-prompt-size.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AAKH,eAAO,MAAM,6BAA6B,IAAI,CAAC;AAE/C,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,wBAAwB;IACvC,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,EAAE,YAAY,EAAE,CAAC;CACvB;AAED,wBAAgB,oBAAoB,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAIzD;AAgCD;;;;GAIG;AACH,wBAAgB,wBAAwB,CACtC,SAAS,EAAE,MAAM,GAChB,wBAAwB,CAiB1B;AAED;;GAEG;AACH,wBAAgB,uCAAuC,CACrD,SAAS,EAAE,MAAM,GAChB,MAAM,CAER"}